Views part 2 – Lincoln Council committees

(This is the second of two posts announcing ScraperWiki “views”. A new feature that Julian, Richard and Tom worked away and secretly launched a couple of months ago. Once you’ve scraped your data, how can you get it out again in just the form you want? See also: Views part 1 – Canadian weather stations.)

Lincoln Council committee updates

Sometimes you don’t want to output a visualisation, but instead some data in a specific form for use by another piece of software. You can think of this as using the ScraperWiki code editor to write the exact API you want on the server where the data is. This saves the person providing the data having to second guess every way someone might want to access it.

Andrew Beekan, who works at Lincoln City Council, has used this to make an RSS feed for their committee meetings. Their CMS software doesn’t have this facility built in, so he has to use a scraper to do it.

First he wrote a scraper in ScraperWiki for a “What’s new” search results page from Lincoln Council’s website. This creates a nice dataset containing the name, date and URL of each committee meeting. Next Andrew made a ScraperWiki view and wrote some Python to output exactly the XML that he wants.

Andrew then wraps the RSS feed in Feedburner for people who want email updates. This is all documented in the Council’s data directory. They used to use Yahoo pipes to do this, but Andrew is finding ScraperWiki easier maintain, even though some knowledge of programming is required.

Since then, Andrew has gone on to make a map for the Lincoln decent homes scheme, also using ScraperWiki views – he’s written a blog post about it.

This entry was posted in developer and tagged , , , , . Bookmark the permalink.

2 Responses to Views part 2 – Lincoln Council committees

  1. Pingback: Views part 1 – Canadian weather stations | Scraperwiki Data Blog

  2. Some great, very useful work. It would be great if Andrew could pass this over to the Lincolnshire County Council as well, who do not have this useful feature (at least for reporters).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s