Rain Prediction for the immediate future using Met Office DataPoint – rain graph

Part of a series of posts to cover some small projects that I did whilst not being able to work. They cover things from the role of familiar strangers on the internet and anti-social networks, through to meteorological hacks, funny memes to twitter bots. This post is about a meteorological hack.

The Met Office in the UK have in this last year published an API for their range of services – all part of the open data movement I think.

DataPoint is a way of accessing freely available Met Office data feeds in a format that is suitable for application developers. It is aimed at professionals, the scientific community and student or amateur developers, in fact anyone looking to re-use Met Office data within their own innovative applications.

The year before this, in Denver, USA, I was shown by a couple of awesome mapping and weather geeks a mobile app that showed when it was going to rain, and more importantly when it wasn’t going to rain in very high temporal resolution. You can use the app and know whether to get a swift half and then leave to get the bus, or whether to stay in for an hour until the showers end. It was very detailed and highly useful. This app Dark Sky App

was freaking awesome. And I want here in the UK, so when the Met Office announced their API I was interested.

You cannot do what DarkSkyApp does with the Met Office DataPoint API though – what you can do is do some interpolations though. The API for precipitation forecasts only gives access to a 3 hourly map tile.


Although further poking around shows that they do have an undocumented 1 hourly image.

Screenshot - 061213 - 16:05:15

These map tiles then could be used. http://rain-graph.herokuapp.com  is the resulting application with the code here: https://github.com/timwaters/rain_graph

It’s a Ruby Sinatra application which for a location, grabs the precipitation tile for a defined location for each hour from now. It looks at the pixel value for the given location and determines the amount predicted. It shows when the heaviest rain is predicted and when it would stop. Interpolation is given by the graph engine itself – no fancy meteorological modelling is done (at this stage). It uses Chunky_png to get the pixel values.


All requests are cached to avoid hitting the MetOffice API and because an image won’t change for an hour. Additionally it uses another API method to get a human readable upcoming forecast text for that location, and displays it under the graph. Contrary to popular global belief it’s not always raining in the UK, and so most of the time you will never see a a graph showing something!


Pixels to Lat Lon:
Since a lat/lon location is quite specific, it could map to one pixel in a tile, and that pixel could have a lower or higher value than the ones surrounding it. I could use a kernel average – do a 6×6 pass over the pixel and get the average value. But since there are tiles are lower zoom levels, by zooming out, the spatial extent of the pixel would equal that larger area – it would do the work for us.

Interpolation between forecasts:
It wasn’t clear if the forecast images showed the predicted situation over the whole hour, or whether it showed the situation at that moment. Should a user look at an animation to see how rain cloud moves across from A->B and guess that in between that there would be rain, or should they think that that there would be no rain if there is no rain shown?

User Interface:
It looks a bit bland – we should show the image tiles underneath  – perhaps shown when hovering over a point.

I haven’t tested the accuracy of this.

Location hard coding:
The text forecasts are hardcoded to a set number of regions, but we could do a closest point and get the correct forecast for the given lat and lon.

Use Yr.No API

Yr.no has detailed hour by hour forecasts API for a place giving the amount of precipitation.


<time from="2013-12-06T19:00:00" to="2013-12-06T20:00:00">
<!-- Valid from 2013-12-06T19:00:00 to 2013-12-06T20:00:00 -->
<symbol number="3" name="Partly cloudy" var="mf/03n.11" />
<precipitation value="0" /><!-- Valid at 2013-12-06T19:00:00 -->
<windDirection deg="294.2" code="WNW" name="West-northwest" />
<windSpeed mps="4.3" name="Gentle breeze" />
<temperature unit="celsius" value="1" />
<pressure unit="hPa" value="1004.9" />

Markov Chains, Twitter and Radical Texts

The next few posts will cover some pet projects that I did whilst not being able to work due to recent civic duty.  They cover things from the role of familiar strangers on the internet and anti-social networks, through to meteorological hacks, funny memes to twitter bots. The first in this series is about what happens when you use markov chains and radical texts with twitter.

Detournement is a technique now considered to the father of remixes or mashups, but with a satirical political nature. Have a look at the wikipedia entry for detournement if you want to know more about it. Basically you do something to something which twists or re routes it so that it makes new meanings. It was the Situationists, led by Debord who really adopted and ran with this as a practice.


Debord would often frequently plagiarise other radical texts in his own work. (The Situationists were also the ones behind original notion of psychogeography – something that you may have caught me talking about before.)

So what would happen if we could detourn, or mashup, or plagiarise Debord’s own writings? And how about if we could publish it periodically, and how about if we had a 140 character limit? Yeah so this is my experiments with these ideas.

Bruna Rizzi; it is from this disastrous exaggeration. The peasant class could not recognize the practical change of products

The proletariat is objectively reinforced by the progressive disappearance of the globe as the bureaucracy can

Markov chains basically work like take a couple of sentences: “A lazy dog likes cheese” and “My house likes to be clean” then look at groups of two or three words together. Then if one of these groups share the same word (“likes”), make a new sentence using that word to chain together. “My house likes cheese” or “A lazy dog likes to be clean”. Markov chains result in sentences that look human readable. The more sentences you feed the population sample, the better or more varied the same of generated sentences.

Some radical texts are complete nonsense and really hard to read, so perhaps applying Markov chains to them can help reveal what truths the obscure language hide.

@markov = MarkyMarkov::TemporaryDictionary.new
@markov.parse_file "debord.txt"
raw_text = @markov.generate_23_words

My solution uses Ruby, the Twitter gem and the marky_markov gem.

https://github.com/timwaters/rattoo  is the work in progress twitter bot – it works currently on Heroku using the scheduler to periodically tweet a sentence, see if any other users have asked it questions and reply back to them.

John Gray on maps and cities.

A map can represent the physical structures of which a city is at any one time composed, but the city itself remains uncharted. This is not only because the city will have changed materially by the time the map appears. A map cannot contain the infinite places that the city contains, which come and go along with the people who pass through them. The map is an abstraction, simplifying experiences that are incomparably more variegated.

From The Silence of Animals. John Gray

Leeds Data Thing, Maps and Hackdays

Leeds Data Thing is a new group started in Leeds  (not to be confused with Leeds Ruby Thing!).

I spoke at the first event (read the write up from Rebecca) about Geospatial visualisations and  OpenStreetMap: Here are the slides:

Since then there has been a few other events as part of Big Data Week – including a load of great short talks.

This weekend there was a data hackday at the UK’s NHS Information Centre for Health and Social Care in the centre of Leeds.

hipster photo

There’s a wealth of data on their website , but it was given to us as a mysql database, and we were able to enter remotely. On the first day I poked around the data and had a thought.


I often spend the first part of any hackday wondering what to do, and twiddling thumbs. I find that hackdays become for me a type of busman’s holiday – and this hackday was particularly geographical in nature. Most of the entries had some kind of data on map component. I think that these types of analyses, whilst being very smart and interesting – and may be exactly what the judges are looking for, may not exactly stretch the unexpected or “the hack” in the data.

Fortunately there was plenty of latitude for exploring things laterally. The most interesting dataset was listing the chemicals and drugs each practice spent money on – but I couldn’t find much to do with it.   What caught my eye was the dataset listing the names of the doctors surgeries, practices, medical centres. If I think about my neighbourhood I can pass about half a dozen doctors in a very small area. Leeds is well covered (or perhaps just my area is!) . I was reminded of James Joyce’s quote about being unable to cross Dublin without passing a pub. Perhaps the same can be said for Leeds and doctors!  The names of the surgeries were also interesting. Names such as:

Chapeloak Surgery
The Avenue Surgery
Dr Ca Hicks’ Practice
The Dekeyser Group Practice
The Highfield Medical Centre
Chapeltown Family Surgery

Wonder if the more “leafy” the name, the more “leafy” the neighbourhood it was in? Perhaps the more grandiose sounding practices had more patients? Perhaps the smaller sounding ones had better patient satisfaction reviews?

At the venue, it appeared that I was the only one to be using Linux on the desktop and so the wifi did not work – so I had a bit over one hour to put something together. Decided to go with the concept of “Leeds is covered” and wanted something showing the labels of the practices over the areas where they were. Filling out the map, so to speak.  The hack was called “Tim’s One Hour Data Challenge” and here is the end result:

Leeds is covered

Leaving GeoIQ/Esri. Retrospective and future plans.

I’ve been with GeoIQ (the folks behind GeoCommons) since the Summer of 2010, and I’ve loved it. Earlier this year GeoIQ joined with Esri and we were hugely excited to change things from the inside and coming up with plans for the new Esri DC Dev team.. However, that’s all behind me now, alas, as it was time to move on. I have left GeoIQ/Esri to be a freelancer and to join the Topomancy coop. This post will take a quick look back at time well spent, and will touch upon what I will be doing in the future.

Continue reading

Pubs in England – geographical distribution of names with cardinal points.

This image shows 4 maps of pubs in England where the pub names have a cardinal direction in the name. North, South, East and West. You can try searching for any pub name here.  For example, West = “The Great Western”, “The Westbourne”. North  = “Northcote Arms”, “The North Pole”

There appears to be more North pubs in London than anywhere else, and more West pubs in the north (and north west?) of England…

Mapwarper.net running on a faster, newer server!

Mapwarper.net has finished maintenance work – I really should get rid of the beta sign now. Anyhow its running on a much faster site, courtesy of Topomancy.com. So performance should be better now. All user accounts and maps and points should have been transferred seamlessly.

mapwarper.net georectify maps georeferencing fun!

I’m still configuring the mail server, so if new users when signing up are seeing mail in spam boxes, let me know!


268 Different Colourful Tiles – Plain Tile Maker

Plain Tile Maker was my weekend project – a mapping tile service that serves one colour tiles as a tile basemap.
There are about 268 colours to choose from – basically anything that the underlying library (Imagemagick) supports.
It was developed for me to play around on the Heroku platform, and as a response to a GeoCommons.com user question, where the user wanted a way to show just a plain one colour background.
This solution is more flexible, and really easy.






The format is


So for example,



And in return you get a 256×256 sized image of that colour.



You can use it in your mapping libraries, for example: With OpenLayers:

var colourTile = new OpenLayers.Layer.XYZ(
 "Plain Colour Tile",
 { sphericalMercator: true,
 buffer: 1,
 numZoomLevels: 17

And with Leaflet:

new L.TileLayer('http://plaintiles.herokuapp.com/colourName/{x}/{y}/{z}.png', {maxZoom: 18});

Or you can add them to GeoCommons quite easily. Add


in the Add a URL link, choosing Map Tile URL from the format, and  then give it a nice name when prompted.

Here’s the Bisque tiles in GeoCommons.

I’ve used the in a map it to give a ghosting effect over the basemap, and to just show my own boundaries. http://geocommons.com/maps/181231  is a map made with some World Boundaries over the plain bisque map we had just added. I’ve also overlaid the Acetate Labels layer on top for context.

Want to know more? The code for this is on github.