Markov Chains, Twitter and Radical Texts

The next few posts will cover some pet projects that I did whilst not being able to work due to recent civic duty.  They cover things from the role of familiar strangers on the internet and anti-social networks, through to meteorological hacks, funny memes to twitter bots. The first in this series is about what happens when you use markov chains and radical texts with twitter.

Detournement is a technique now considered to the father of remixes or mashups, but with a satirical political nature. Have a look at the wikipedia entry for detournement if you want to know more about it. Basically you do something to something which twists or re routes it so that it makes new meanings. It was the Situationists, led by Debord who really adopted and ran with this as a practice.

guy_debord1

Debord would often frequently plagiarise other radical texts in his own work. (The Situationists were also the ones behind original notion of psychogeography – something that you may have caught me talking about before.)

So what would happen if we could detourn, or mashup, or plagiarise Debord’s own writings? And how about if we could publish it periodically, and how about if we had a 140 character limit? Yeah so this is my experiments with these ideas.

Bruna Rizzi; it is from this disastrous exaggeration. The peasant class could not recognize the practical change of products

The proletariat is objectively reinforced by the progressive disappearance of the globe as the bureaucracy can

Markov chains basically work like take a couple of sentences: “A lazy dog likes cheese” and “My house likes to be clean” then look at groups of two or three words together. Then if one of these groups share the same word (“likes”), make a new sentence using that word to chain together. “My house likes cheese” or “A lazy dog likes to be clean”. Markov chains result in sentences that look human readable. The more sentences you feed the population sample, the better or more varied the same of generated sentences.

Some radical texts are complete nonsense and really hard to read, so perhaps applying Markov chains to them can help reveal what truths the obscure language hide.

@markov = MarkyMarkov::TemporaryDictionary.new
@markov.parse_file "debord.txt"
raw_text = @markov.generate_23_words

My solution uses Ruby, the Twitter gem and the marky_markov gem.

https://github.com/timwaters/rattoo  is the work in progress twitter bot – it works currently on Heroku using the scheduler to periodically tweet a sentence, see if any other users have asked it questions and reply back to them.

John Gray on maps and cities.

A map can represent the physical structures of which a city is at any one time composed, but the city itself remains uncharted. This is not only because the city will have changed materially by the time the map appears. A map cannot contain the infinite places that the city contains, which come and go along with the people who pass through them. The map is an abstraction, simplifying experiences that are incomparably more variegated.

From The Silence of Animals. John Gray

Leeds Data Thing, Maps and Hackdays

Leeds Data Thing is a new group started in Leeds  (not to be confused with Leeds Ruby Thing!).

I spoke at the first event (read the write up from Rebecca) about Geospatial visualisations and  OpenStreetMap: Here are the slides:

Since then there has been a few other events as part of Big Data Week – including a load of great short talks.

This weekend there was a data hackday at the UK’s NHS Information Centre for Health and Social Care in the centre of Leeds.

hipster photo

There’s a wealth of data on their website , but it was given to us as a mysql database, and we were able to enter remotely. On the first day I poked around the data and had a thought.

Hackdays

I often spend the first part of any hackday wondering what to do, and twiddling thumbs. I find that hackdays become for me a type of busman’s holiday – and this hackday was particularly geographical in nature. Most of the entries had some kind of data on map component. I think that these types of analyses, whilst being very smart and interesting – and may be exactly what the judges are looking for, may not exactly stretch the unexpected or “the hack” in the data.

Fortunately there was plenty of latitude for exploring things laterally. The most interesting dataset was listing the chemicals and drugs each practice spent money on – but I couldn’t find much to do with it.   What caught my eye was the dataset listing the names of the doctors surgeries, practices, medical centres. If I think about my neighbourhood I can pass about half a dozen doctors in a very small area. Leeds is well covered (or perhaps just my area is!) . I was reminded of James Joyce’s quote about being unable to cross Dublin without passing a pub. Perhaps the same can be said for Leeds and doctors!  The names of the surgeries were also interesting. Names such as:

Chapeloak Surgery
The Avenue Surgery
Dr Ca Hicks’ Practice
The Dekeyser Group Practice
The Highfield Medical Centre
Chapeltown Family Surgery

Wonder if the more “leafy” the name, the more “leafy” the neighbourhood it was in? Perhaps the more grandiose sounding practices had more patients? Perhaps the smaller sounding ones had better patient satisfaction reviews?

At the venue, it appeared that I was the only one to be using Linux on the desktop and so the wifi did not work – so I had a bit over one hour to put something together. Decided to go with the concept of “Leeds is covered” and wanted something showing the labels of the practices over the areas where they were. Filling out the map, so to speak.  The hack was called “Tim’s One Hour Data Challenge” and here is the end result:

Leeds is covered

WhereCampEU 2011 recap & berlin psychogeography

Last week in Berlin I was lucky enough to go to WhereCampEU – thanks to Gary and Chris for organising this wonderful unconference. The conference was held in a trendy hipster ish part of the city, but which had also, I heard, the highest number of young families and births. It was also in the former Eastern part of the city. It gave the area a nice appeal, overall.

photo by Chris Fleming

I did a couple of sessions, one on a preview of GeoCommons2.0  talked about in a previous post and the other a psychogeography session. For the psychogeography session I sent four teams out to explore the environs around the campus.

One team followed people around. They said “I’m amazed by how slowly some people moved” and “Well, often we followed someone and then they would wander into a book shop”  – revealing the nature of the people and the type of area, bohem style cafes and shops, lazily people.

Another group were sent to ask people to point to were the centre of Berlin was. I asked some people where they thought was the centre, and most of them scratched their chins, and pointed to the Mitte area of the city, usually on the map, or waved southwards. Part of a consequence of being a split city, really. The western bit, someone said, “looks and feels more like a CBD” – that is, big shops, tall towers etc. I did venture to the former western CBD centre, and came across a mile long car show. This area was where the money was.

The other group was sent to walk around the area according to the Game of Life algorithm, Left left Right where you walk and take the first left, then the second left, then the next right, and so on. It’s impossible to predict where you will end up. I joined this group. We had a good explore over a small area, really, but encountering a lot of different environments. Shared (private) gardens / courtyards in the middle of apartment blocks, churches, cafes, and shops.

The fourth team were given a secret mission, and so I cannot reveal to you what they did. However, they are all in good health, and saw the city in a new light.

Photo by Chris Fleming

Back to the unconference, and some of the highlights were:

* Playing the Skobbler game, treasure hunting for addresses in the neighbourhood.

* Seeing offmaps evolve over the year. I’ve not got an iPhone, but that app looked very nice.

* Spatial databases, and in particular CouchDb – and their spatial bits

* CASA did a few talks – I’m getting more and more fond of their work – if anything they really seem to love the stuff they are doing – they share the same vision as me as giving GI tools and benefits to as many people as possible.

* Peter Batty wore an ipad t-shirt – and gave a great presentation about essentially putting utilities information onto a Gmaps like interface and mobile map.

* Gary Gale gave a compelling reason for standardizing place. And it makes sense.

* Meeting the NomadLabs guys for the first time, and being able to say “Thank You” for their work on Ruby on Rails GIS Hacks that I found very useful 4 years ago!

* Corridor talk, beer and food

Henry Miller on Seeing a Place

From “The Eye of Paris” in The Wisdom of the Heart – Henry Miller, 1941, New Directions:

Now and then, in wandering through the streets, suddenly one comes awake, perceives with a strange exultation that he is moving through an absolutely fresh sliver of reality. Everything has the quality of the marvelous – the murky windows, the rain-sodden vegetables, the contours of the houses, the bill-posters, the slumping figures of men and women, the tin soldiers in the stationery shops,  the colors of the walls – everything written down in an unfamiliar script. After the moment of ecstasy has passed what is one’s amazement but to discover that the street through which he is walking with eyes popping is the street on which he lives. He has simply come upon it unaware, from the wrong end perhaps. Or, moving out of the confines of an unknown region, the sense of wonder and mystery prolonged itself in defiance of reality. It is as if the eye itself had been freshened, as if it had forgotten all that it had been taught. In this condition it happens that one really does see things he had never seen before –  not the fantastic, harrowing, hallucinating objects of dream or drug, but the most banal, the most commonplace things, seen as it were for the first time.

Sticks & Booze – Beating the Bounds in Headingley

About 25 people turned up for a Beating the Bounds psychogeographical walk around the Headingley (Leeds, UK) DPPO Boundary.  It was run by me and the Leeds Psychgeography Group. Tina runs and blogs about the group, and there’s a Leeds Psychogeography Group Facebook Page to boot.  All photos here were taken by Mark Jaffé, cheers!

A DPPO stands for a Designated Public Place Order. Essentially within that area, if you are causing a nuisance or annoyance, a police constable can stop you drinking, confiscate booze, up end cans etc. If you don’t comply then that is when you may be breaking the law. It’s a law to stop street drinkers mainly. In the Hyde Park and Woodhouse DPPO area, well over 80%  (over 300 in a few months) of people stopped, were students. Phil Kirby went on the first of the Beating the Bounds walks, and has blogged about it on The Culture Vulture.

We were to talk widdershins, anti-clockwise around the boundary – a magical act, designed to disperse any energies or what have you. I brought along some sticks (loop cane, chopped in half) for people to beat the ground at certain points. Oh, and we drank as we went – essentially beating the restrictions.

We passed through some nice areas – here on the ridge, was the Wassailing tree. We also encountered the strangest plot of land in Headingley, fish and chips and had assorted adventutres.

Games developed, whereby when a DPPO warning sign was found, a drink had to be taken.

It took about 2 hours. I was quite tipsy at the end. in the above picture, we can see a stick being used in it’s traditional role of beating a boy.

I’ll be doing some more psychgeography posts in time. The next beating the bounds walk will go around the city centre – it’s a huge area!