State of the Map Europe 2014 – Pure OpenStreetMap.

Karlsruhe

State of the Map Europe 2014 was in the German city of Karlsruhe. The city was a planned city – designed and built around 1715 – pre motor car, but with wide avenues, and half of the city seems to be a park. It’s also famous for being the home of the Karlsruhe Addressing Scheme - an example of a folksonomy tagging convention that everyone pointed to and adopted, due to the great mappers there – including the folks from Geofabrik.de who also organised the conference. Here are some notes from the conference:

Nature of the conference

The European conference seemed much more intimate with a focus on developer and contributors  - compared to the US Conference which I think had more end users and people sent there by their bosses for their company. Pretty much every single session was on topic (except for the closing buzzword laden keynote!)  - and as such there were no enlightening talks about psychogeography, general historical mapping, or other geospatial software. It was pure OSM.

All the talks are online and the video recordings are on youtube and I encourage you to view them.

3D maps

3D Maps, such as Mapzen and OSMBuildings were prominent – and both showed off some very creative ways of representing 3D maps.

Geocoder and Gazetteers

The only track in the conference – this was full of gazetteers with an announcement from OpenCage and MapZen – all appear to be using ElasticSearch – same as we (Topomancy) did last year for the NYPL and Library of Congress. Check out gazetteer here.

Other stuff

Trees – Jerry did a talk about mapping trees – about how they were represented in historical maps previously, and how we can use SVG symbols to display woods and trees in a better way. Jerry lead an expedition and workshop on the morning of the hack day to show participants the different habitats, surface types and variance in the environment that mappers could take into consideration.

Mapbox WebGL – Constantine, a European engineer of Mapbox did a fascinating talk about the complexities of the technical challenges with vector tiles and 3D maps. I really enjoyed the talk.

Image

OpenGeoFiction - using the OSM stack to create fictional worlds  - not fantasy or science fiction, but amazing experiments in amateur planning, utopian visions and creative map making. OpenGeoFiction.net

The fictional world of Opengeofiction is thought to be in modern times. So it doesn’t have orcs or elves, but rather power plants, motorways and housing projects. But also picturesque old towns, beautiful national parks and lonely beaches.

I love this project!

Vector Tiles – Andy Allan talked about his new vector tile software solution ThunderForest – being one of the only people to know the ins and outs of how Mapbox do the Mapnik / TileMill vector magic. ThunderForest powers the cycle map. Vector maps has lots of advantages and I think we’d probably use it for OpenHistoricalMap purposes at some stage. Contact Andy for your vector mapping and online cartographic needs!

POI Checker – from the same house as WheelMap.org comes POI Checker – it allows organisations to compare their data with data in OSM  - and gives a very neat diff view of Points of Interests. This could be a good project to follow.

Historical Stuff

OpenHistoricalMap There were a few things about historical maps in the conference, although in my opinion less than at any other SOTM previously. I did a lightning talk about OpenHistoricalMap and completely failed to mention the cool custom UK centric version of the NYPL’s Building Inspector.

Opening Keynote  - this was peppered with the history of the city and gave a number of beautiful historical map examples. Watch the video.

Map Roulette v2 – Serge gave a talk about the new version of Map Roulette  - it is being customised to be able to run almost any custom task on the system. We chatted a the hack day to see if the tasks from the Building Inspector could be a good fit into the new Map Roulette – I will look into this!

 

 

Rain Prediction for the immediate future using Met Office DataPoint – rain graph

Part of a series of posts to cover some small projects that I did whilst not being able to work. They cover things from the role of familiar strangers on the internet and anti-social networks, through to meteorological hacks, funny memes to twitter bots. This post is about a meteorological hack.

The Met Office in the UK have in this last year published an API for their range of services – all part of the open data movement I think.

DataPoint is a way of accessing freely available Met Office data feeds in a format that is suitable for application developers. It is aimed at professionals, the scientific community and student or amateur developers, in fact anyone looking to re-use Met Office data within their own innovative applications.

The year before this, in Denver, USA, I was shown by a couple of awesome mapping and weather geeks a mobile app that showed when it was going to rain, and more importantly when it wasn’t going to rain in very high temporal resolution. You can use the app and know whether to get a swift half and then leave to get the bus, or whether to stay in for an hour until the showers end. It was very detailed and highly useful. This app Dark Sky App

was freaking awesome. And I want here in the UK, so when the Met Office announced their API I was interested.

You cannot do what DarkSkyApp does with the Met Office DataPoint API though – what you can do is do some interpolations though. The API for precipitation forecasts only gives access to a 3 hourly map tile.

http://www.metoffice.gov.uk/datapoint/product/precipitation-forecast-map-layer

Although further poking around shows that they do have an undocumented 1 hourly image.

Screenshot - 061213 - 16:05:15

These map tiles then could be used. http://rain-graph.herokuapp.com  is the resulting application with the code here: https://github.com/timwaters/rain_graph

It’s a Ruby Sinatra application which for a location, grabs the precipitation tile for a defined location for each hour from now. It looks at the pixel value for the given location and determines the amount predicted. It shows when the heaviest rain is predicted and when it would stop. Interpolation is given by the graph engine itself – no fancy meteorological modelling is done (at this stage). It uses Chunky_png to get the pixel values.

687474703a2f2f692e696d6775722e636f6d2f4343306f5868772e706e67

All requests are cached to avoid hitting the MetOffice API and because an image won’t change for an hour. Additionally it uses another API method to get a human readable upcoming forecast text for that location, and displays it under the graph. Contrary to popular global belief it’s not always raining in the UK, and so most of the time you will never see a a graph showing something!

Considerations:

Pixels to Lat Lon:
Since a lat/lon location is quite specific, it could map to one pixel in a tile, and that pixel could have a lower or higher value than the ones surrounding it. I could use a kernel average – do a 6×6 pass over the pixel and get the average value. But since there are tiles are lower zoom levels, by zooming out, the spatial extent of the pixel would equal that larger area – it would do the work for us.

Interpolation between forecasts:
It wasn’t clear if the forecast images showed the predicted situation over the whole hour, or whether it showed the situation at that moment. Should a user look at an animation to see how rain cloud moves across from A->B and guess that in between that there would be rain, or should they think that that there would be no rain if there is no rain shown?

User Interface:
It looks a bit bland – we should show the image tiles underneath  - perhaps shown when hovering over a point.

Accuracy:
I haven’t tested the accuracy of this.

Location hard coding:
The text forecasts are hardcoded to a set number of regions, but we could do a closest point and get the correct forecast for the given lat and lon.

Use Yr.No API

Yr.no has detailed hour by hour forecasts API for a place giving the amount of precipitation.

http://www.yr.no/place/United_Kingdom/England/Leeds/hour_by_hour_detailed.html

<time from="2013-12-06T19:00:00" to="2013-12-06T20:00:00">
<!-- Valid from 2013-12-06T19:00:00 to 2013-12-06T20:00:00 -->
<symbol number="3" name="Partly cloudy" var="mf/03n.11" />
<precipitation value="0" /><!-- Valid at 2013-12-06T19:00:00 -->
<windDirection deg="294.2" code="WNW" name="West-northwest" />
<windSpeed mps="4.3" name="Gentle breeze" />
<temperature unit="celsius" value="1" />
<pressure unit="hPa" value="1004.9" />
</time>

Mapwarper.net running on a faster, newer server!

Mapwarper.net has finished maintenance work – I really should get rid of the beta sign now. Anyhow its running on a much faster site, courtesy of Topomancy.com. So performance should be better now. All user accounts and maps and points should have been transferred seamlessly.

mapwarper.net georectify maps georeferencing fun!

I’m still configuring the mail server, so if new users when signing up are seeing mail in spam boxes, let me know!

Cheers!

GeoCommons GeoJSON in OpenLayers

So, with GeoCommons you can export all the features from a dataset in GeoJSON format. This is very useful. Then can it be displayed in Openlayers? Why yes it can!

I use the following strategy, wrapping the response with a little bit extra so that the GeoJSON format can read it properly.

var url = "http://geocommons.com/overlays/128725/features.json?limit=100";

var p = new OpenLayers.Format.GeoJSON();

OpenLayers.loadURL(url, {}, null, function (response) {
 var gformat = new OpenLayers.Format.GeoJSON();
 gg = '{"type":"FeatureCollection", "features":' +
      response.responseText + '}';
 var feats = gformat.read(gg);

 vector_layer.addFeatures(feats);
});

Have a look a the live demo of this example of loading points from a Geocommons dataset straight into an OpenLayers map
(Note that I am using an OpenLayers.ProxyHost proxy to make it work in FF)

It is of course very basic in terms of styling, but it’s a start!

Incidentally the points are from “CARMA, India Power Plant Emissions, India, 2000/ 2007/Future” (carbon monitoring)

WhereCampEU 2011 recap & berlin psychogeography

Last week in Berlin I was lucky enough to go to WhereCampEU – thanks to Gary and Chris for organising this wonderful unconference. The conference was held in a trendy hipster ish part of the city, but which had also, I heard, the highest number of young families and births. It was also in the former Eastern part of the city. It gave the area a nice appeal, overall.

photo by Chris Fleming

I did a couple of sessions, one on a preview of GeoCommons2.0  talked about in a previous post and the other a psychogeography session. For the psychogeography session I sent four teams out to explore the environs around the campus.

One team followed people around. They said “I’m amazed by how slowly some people moved” and “Well, often we followed someone and then they would wander into a book shop”  - revealing the nature of the people and the type of area, bohem style cafes and shops, lazily people.

Another group were sent to ask people to point to were the centre of Berlin was. I asked some people where they thought was the centre, and most of them scratched their chins, and pointed to the Mitte area of the city, usually on the map, or waved southwards. Part of a consequence of being a split city, really. The western bit, someone said, “looks and feels more like a CBD” – that is, big shops, tall towers etc. I did venture to the former western CBD centre, and came across a mile long car show. This area was where the money was.

The other group was sent to walk around the area according to the Game of Life algorithm, Left left Right where you walk and take the first left, then the second left, then the next right, and so on. It’s impossible to predict where you will end up. I joined this group. We had a good explore over a small area, really, but encountering a lot of different environments. Shared (private) gardens / courtyards in the middle of apartment blocks, churches, cafes, and shops.

The fourth team were given a secret mission, and so I cannot reveal to you what they did. However, they are all in good health, and saw the city in a new light.

Photo by Chris Fleming

Back to the unconference, and some of the highlights were:

* Playing the Skobbler game, treasure hunting for addresses in the neighbourhood.

* Seeing offmaps evolve over the year. I’ve not got an iPhone, but that app looked very nice.

* Spatial databases, and in particular CouchDb – and their spatial bits

* CASA did a few talks – I’m getting more and more fond of their work – if anything they really seem to love the stuff they are doing – they share the same vision as me as giving GI tools and benefits to as many people as possible.

* Peter Batty wore an ipad t-shirt – and gave a great presentation about essentially putting utilities information onto a Gmaps like interface and mobile map.

* Gary Gale gave a compelling reason for standardizing place. And it makes sense.

* Meeting the NomadLabs guys for the first time, and being able to say “Thank You” for their work on Ruby on Rails GIS Hacks that I found very useful 4 years ago!

* Corridor talk, beer and food

Some stuff I’ve been working on with new GeoCommons 2.0

Last weekend, if you were at WhereCampEU in Berlin (blog post to follow) , you may have caught my sneak peak into the new GeoCommons 2.0, which has been revealed just the other day. Here are some of the highlights of the new GeoCommons

  • The flash map has been overhauled and re-written, mainly by Andrei – and it can handle hundreds of thousands numbers of points quite happily
  • Analytics library is completed, but not currently accessible to normal users of GeoCommons – hopefully it will be soon, if people want it.
  • Behind the scenes, the system uses a number of distributed workers and tasks to offload processing intensive or long processing tasks
  • Datasets and Maps get given nice overview images, and the attributes of datasets have histograms generated for them
  • Data can be edited in the system, and filtered, and saved either to replace itself or as a new dataset
  • Animation of temporal data is much nicer now
  • Polymaps for HTML5 non-flash map support
  • Filters can be applied to the map, so that attributes can be filtered out.
  • Thematic maps can be made with categories now
  • Acetate is used as standard
  • Custom markers can be added to a map, and even animated ones work too!
The GeoIQ developer blog has a developer orientated review of wha’ts new and there is a good overview of GeoCommons on the main GeoIQ blog too.
Keep your eyes peeled on the GeoIQ Developer Blog over the next few days as the team adds some more posts about some of the technology behind it.

WherecampUK 2010 Recap

Last week I journeyed down on the train to Nottingham to go to WhereCampUK – an unconference for all things “geo” – but it was only a few months since the similarly named WhereCampEU of (which I never actually wrote something about) down in London. Before I share some of the best bits, here’s some of the similarities and differences.

* Less international folks

* Less big geo personalities and keynotes

* More OSM

* No T-Shirts

* More beer – we drank a large pub dry, literally. The next day, the landlord swore at me for pissing off their regulars.

* More cake

* Cheaper and quicker to run, setup and organise.

For more pointers in how to run an unconference, check out Steve Coast‘s latest post, where he writes about what he did for Wherecamp in Denver.  How I ran a successful unconference in 6 hours and you can too.

Overall, the event was great.

I ran two sessions. The main one was “What is Psychogeography“. The best part of this was sending all participants out with directions in twos and threes, for 10 mins before lunch. They had directions such as “left left right”, “follow someone”, “ask where the centre is, follow that direction, ask again”, “find hidden portals”, “find fairies”, “hear something, take a photo”.

I also quickly slotted in the NYPL Warper presentation, and included this slide. You get 20 points if you know what this refers to!

I also mentioned the words “neogeography” for the first time in the conference, and that was at 3:30pm, which says quite a bit about the use of the term.

Talks I liked were:

* Vernacular Geography & Informal Placenames

* Geo Games

* Education and mobile maps

* Augmented Reality roundup

* How streets get names

* Peoples Collection Wales

* Haptic Navigation

* OSM Talks including – Potlatch 2

* Gregory Mahler’s – I’m a Psycho Mapper!

* OSGEO



AGI GeoCommunity – NYPL & Mapwarper

Yesterday I presented at the AGI Annual conference on the NYPL Map Rectifier. I was also able to launch MapWarper.net – to showcase the user submitted version of the code base, which you can get on github, should you like! I’ll put the slides up to slideshare asasp

I also showed the video of the newest Stamen Design work on the redesign of the interface. This interface will be applied to the application, and will be featuring at the British Library Growing Knowledge exhibition in a couple of weeks

The whole conference was okay, well organised, and well attended by fellow AGI Northern Group members. The soapbox – an evening even over beers – short 20:20′s but with the emphasis on ranting and comedy proved very popular, and was very amusing – keep an eye out for the videos when they come out!

The free W3Geo unconference which was on the day before the main conference, and as Ed writes, had the more interesting talks and energy here. w3g had a number of fixed keynotes and speakers, which I think was a good idea – it allowed people to attend knowing that certain people would be speaking. Alas, I didn’t really believe it was promoted that well beforehand.

The main conference had a number of interesting talks. The guys from CASA really do seem to be enjoying themselves. Survey Mapper was highlighted – essentially it’s an onlne survey application, but with, yes geography! It would be great to be able to allow the export of the results from a survey, and use this as a data layer in GeoCommons – to compare against any number of other datasets. They also showcased their work with Twitter – exploratory – one thing with pretty much all twitter map experiements is that people are so overwhelmed with the dataset, that the analysis, the task of generating information from this data tends to be overlooked. Most of the mapping done tends to show that, for the very first time, that people tend to live and do things in urban areas.

ESRI were keynoting, but with a somewhat downbeat “why doesn’t anyone understand and use GIS” cry, with a call to promote the usefullness of GIS to decision makers. Nothing new here, but in these economic times, seemed to imply that GIS budgets were being hit hard.

Nigel Shadbolt’s closing talk convinced me of the importance of linked data. However, the talk was more about the benefits and importance of Open Data as a whole, and he was speaking at such a higher level than most of the other talks were operating at. They were talking about “how to share” data, whilst with open data at a national level operates at a much more fundamental level

The Ordnance Survey have now escaped much of the fights and shouting what with the OS Open Data, and today have announced better, clearer, more generous terms for Derivative data (see here and here for more info on that

Haiti #2

Almost exactly a year ago, I finished the import of a large amount of road data for Haiti. This was the result of the mapping community responding to the devastating storms and flooding that shook the country, and the need for good quality data for people to share.  Now the country is struck by another disaster.

The main page coordinating efforts from OSM is here:

http://wiki.openstreetmap.org/wiki/WikiProject_Haiti#2010_Earthquake_Response

We’ve set up a special Map Warper instance at  http://maps.nypl.org/relief/maps has a number of public domain maps that you can warp. Please come along and rectify these maps. OSM volunteers will then trace over the top and add these onto the map.

WhooMS – a tiny public geotiff WMS server

WhooMS is a tiny public WMS server for those people who have a GeoTIFF and need someplace to serve it as WMS. I wrote it a while ago, but now its running for all to use.

It’s written in Ruby, using the Sinatra Web Framework, which basically means it can all fit neatly on one file.  It uses Ruby Mapscript to read the uploaded GeoTiff and serve it out to the world.

Nice and simple and basic. Got a GeoTiff handy? (EPSG:4326) give it a go. whooms.mapwarper.net

The code for it can be found on github: http://github.com/timwaters/whooms

Main caveat: when the disc space gets full, the older files will be deleted to fill up space.