Markov Chains, Twitter and Radical Texts

The next few posts will cover some pet projects that I did whilst not being able to work due to recent civic duty.  They cover things from the role of familiar strangers on the internet and anti-social networks, through to meteorological hacks, funny memes to twitter bots. The first in this series is about what happens when you use markov chains and radical texts with twitter.

Detournement is a technique now considered to the father of remixes or mashups, but with a satirical political nature. Have a look at the wikipedia entry for detournement if you want to know more about it. Basically you do something to something which twists or re routes it so that it makes new meanings. It was the Situationists, led by Debord who really adopted and ran with this as a practice.


Debord would often frequently plagiarise other radical texts in his own work. (The Situationists were also the ones behind original notion of psychogeography – something that you may have caught me talking about before.)

So what would happen if we could detourn, or mashup, or plagiarise Debord’s own writings? And how about if we could publish it periodically, and how about if we had a 140 character limit? Yeah so this is my experiments with these ideas.

Bruna Rizzi; it is from this disastrous exaggeration. The peasant class could not recognize the practical change of products

The proletariat is objectively reinforced by the progressive disappearance of the globe as the bureaucracy can

Markov chains basically work like take a couple of sentences: “A lazy dog likes cheese” and “My house likes to be clean” then look at groups of two or three words together. Then if one of these groups share the same word (“likes”), make a new sentence using that word to chain together. “My house likes cheese” or “A lazy dog likes to be clean”. Markov chains result in sentences that look human readable. The more sentences you feed the population sample, the better or more varied the same of generated sentences.

Some radical texts are complete nonsense and really hard to read, so perhaps applying Markov chains to them can help reveal what truths the obscure language hide.

@markov =
@markov.parse_file "debord.txt"
raw_text = @markov.generate_23_words

My solution uses Ruby, the Twitter gem and the marky_markov gem.  is the work in progress twitter bot – it works currently on Heroku using the scheduler to periodically tweet a sentence, see if any other users have asked it questions and reply back to them.

With GeoCommons

I’m now working with FortiusOne and GeoCommons!

Which will mean me working on lots of exciting geo stuff, and you getting to play with it on soon!

Can’t really get into much detail at this time except that GeoCommons shares the same vision of making geospatial data, tools and analysis available for all to use, through the web – the same vision which got me started freelance in the first place, so I’m very enthusiastic.

I’ll still be supporting and developing on and the NYPL Warper – and on that note, expect some eye popping news from Stamen Design related to this soon!  In the meantime have a play with

using RAM as chromium cache for speed

Just a little post to help other folks find this hint and tip. I’m using Ubuntu on an Acer Aspire One with the SSD – I recommend fellow Ruby allied Leodian George’s post on getting it all set up if you are interested.

Okay, to make Chromium or Google Chrome to go very fast and use a previously configured RAM filesystem (I used aufs). Use the –disk-cache-dir flag. For example, where /var/tmp is actually held in RAM:

chromium-browser --disk-cache-dir="/var/tmp"

Will get the browser using a cache in RAM, and not the actual SSD. Hope this helps others out there.

Lazy Apps at Barcamp Bradford

Bradford had a barcamp at last, and it was rather good. What was telling was that every single slot was booked out, and repeats of some sessions were requested, and shoved in during breaks.

Cheers to all the folks who organized and sponsored it etc. Ian gives a good summary about what happened.

“Lazy Apps” was the title of my session, and as the room it was in was furthest from the wifi, we didn’t live stream them to twitter. Instead we lazily scribbled them on paper. Here they are for you.

pre update update

Just a quick note to say that I’ve been to and will be blogging about:

  • O’Reilly’s Where2.0 in San Jose, USA
  • Wherecamp 2009 in Palo Alto, USA
  • Humanitarian & geo coworking, San Francisco, USA
  • Maker Faire in San Mateo, USA
  • Alertnet’s Mapping Workshop / Webinar, in London, UK
  • Pateley Bridge OpenStreetMap Mapping Party, North Yorkshire, UK
  • Unsheffield (Barcamp), in Sheffield, UK
  • First UK Open Source GIS Conference in Nottingham, UK.

So that’s quite a lot of stuff for my brain to digest… expect to see a stream of posts starting shortly.

The shell meme.

Found The Shell Meme over on Caius’s blog and, since it’s a meme that’s entirely voluntary, and doesn’t require me to actively spam other people for them to do it, here it is:
Type this into a new terminal / shell:
history|awk '{a[$2]++} END{for(i in a){printf "%5d\t%s\n ",a[i],i}}'|sort -rn|head

and I got this:
79 svn
53 ls
52 rake
30 cd
25 identify
25 convert
21 gdalinfo
21 cucumber
18 gnome-open
15 rm

All in preparation of my newer Map Warper which will come out very soon.

bbc radio shell script

You may find this useful. Shell script for listening to bbc radio feeds via mplayer, nice and short. “bbc 1″ plays bbc 1.
list of feed urls:
if [ $# -eq 0 ]
echo "usage bbc 1"
for ((i=0; i<${#bbc[@]}; i++))
echo $i . ${bbc[$i]}
exit 1
mplayer -quiet -playlist ${bbc[${1}]}