#NMBS twitter analysis for the last 30 days.

A picture says a thousand words, they say. So here is one that shows how many tweets there were for #nmbs the last 30 days. For those of you outside Belgium, the nmbs is the flemish name for the Belgian Trainroad company. #SNCB is the french name for the same company.

(I pulled this graph and the tweet list below from my Twitalytics program that I’m working on…)

last 3O days of twitter activity for the hashtag #nmbs

With this picture you immediately notice how May 26 seems especially active. Let’s see what sort of secondary keywords people where using that day in conjunction with #nmbs:

#nmbs(69) #fail(13) #news(5) #schaarbeek(3) #fb(3) #bxl(3) #breaking(3) #bommelding(3) #terroralert(2) #noord(2) #eu(2)

We can already deduce from these keywords (and my memory of the day) that there was a bomb alert that put the whole train system out of order for several hours.

People were not happy. Out of the 69 tweets for that day about the keyword #nmbs, 13 include the secondary keyword #fail. That’s 18% percent of the tweets that can immediately be considered negative ! It’s actually more, if you check the tweets themselves.

Here’s a small extract of what the people were tweeting (this is in flemish mostly):

32398 + willemsstijn (Stijn Willems) – 2010-05-26 17:38:36

Chaos in Brussel #NMBS

32401 + crispkat (Katrien Crispeyn) – 2010-05-26 17:12:48

In Leuven dan geen personeel voor trein richting Heverlee #fail #nmbs

32404 + mhitschf (Maureen Hitschfeld) – 2010-05-26 17:09:39

What I dislike the most is the lack of information! All trains delayed and no explanation at all! #NMBS

32407 + dubtje (Jan Debonnet) – 2010-05-26 17:05:38

Als de #nmbs nu een beetje zijn best doet kan ik m’n aansluiting missen en nog een uurtje ronddraaien in brugge.

32411 + jacobvb (Jacob) – 2010-05-26 17:00:59

zucht … #nmbs #fail AGAIN! http://tweetphoto.com/24050442

32413 + miespaties (Marlies Deforche) – 2010-05-26 16:52:40

En zo een MIVB bus ontdekt v Bxl Noord nr Vilvoorde, #58 #nmbs #bommelding #hoppa

32415 + avaneech (AnneliesVaneechoutte) – 2010-05-26 16:50:39

Wat een slagveld #brusselnoord #nmbs

32416 + jnloco (Jelle Loquet) – 2010-05-26 16:39:38

RT @isabelleminnebo: Bxl Noord is nu een beetje zoals de Titanic: veel te veel mensen, agressiviteit, paniek en iedereen wil hier gewoon WEG! #bxl #nmbs #noord

32418 + jnloco (Jelle Loquet) – 2010-05-26 16:39:01

Niettegenstaande bommelding in Schaarbeek, thuisgeraakt met een minieme vertraging van 15′. #gelukzak #nmbs

32422 + bartnelis (Bart Nelis) – 2010-05-26 16:34:21

Hallo!! Mogen de reizigers voor turnhout ook eens weten waar ze aan toe zijn ? Kloothommels #nmbs #fail

It’s nice to have a timeline view on how many tweets per day. Graphs do tell more than just words and numbers…It actually gave me some more ideas to implement !

[Update:  I have added a demo page to my little app that follows the #nmbs keyword]

Sparklines are cool !

Recently I was looking to add some graphics to my python CherryPy application, and I discovered sparklines.

Sparklines are small graphical displays of numerical data that help you to get a timeline view. Their size makes it easy to add them in phrases or to compare different graphs quickly. Below are some examples:

Examples of the sparkline graphics

This javascript jquery plugin is simplicity itself :

  1. Add the js script to your html page and add a jquery function
  2. Generate some numbers between a span tag
  3. And give that span a jquery function classname

And bob’s your uncle !

Go discover it for yourself !

Twitter announces annotations – A marketeers wet dream.

I’ve started reading up on the new announcement from Twitter on Annotations – excuse me, I am behind on my reading schedule with all this programming recently. I started writing this post on May 9th, but I got caught up in wrestling python datetime.datetime deamons, their minion timedelta and the unholy union they made with SQLObject …

This blogpost on Sitepoint gives a good point of view on it means for the average user and discusses the possible impacts of using the annotations system. Do read the comments as well ! There are others as well.

In a nutshell, annotations seem to be an answer to save space (more of those precious 140 characters for your actual message) and create a way to more semantically structure the meta-data that is contained inside the message.

I think most of normal users will happily continue on using the old-style hash-marks and special signs. It’s just easier to do so, just type what you are thinking.

However, Twitter is not doing this for the normal people. It’s doing it to make money, it’s doing this for the businesses on Twitter.

Twitter *has* to make this modification if it wants to start attracting advertisers and their money.

Businesses will simply love the fact that, while helping the customer, they can also include a payload of  relevant information:

  • I’m working for a car dealer: if I’m responding to any tweets, I would love the opportunity to add links to car descriptions on my website, relevant pricing info, etc to you. Maybe even a small image of the car ?
  • Ask a quote of an online insurance company via their website or even via a direct tweet: you get a tweet back, containing:
    1. a quote-in-a-tweet,
    2. the link to the full pdf file to download,
    3. if you enabled location-based tweets, the Google map link to the nearest agency in your location,
    4. the next steps that you need to do to get started…

    Heck, you could make a small mini flow/website out of your tweet  if you could use javascript and a compatible client !

  • baseball/football statistics: somebody is tweeting live coverage of a game that is going on. For any goal (excuse me, I only know football very well) that is made you receive a tweet when this happens, with the players statististics (score average, last time scored,…)

If it works out and developers can figure out a common way of describing metadata, Annotations will be something very nice for a certain type of twitter user (the businesses). I just don’t think hashtags will disappear any time soon.

Compiling MySQLdb for Python on mac – djeez !

This seriously needs some attention from someone with some config knowledge. Me and a horde of other python developers who transition to mac have stumbled and failed miserably on setting the python msql adapter on the mac.

Instead of using easy_install like most other python pieces you need for your project, you need to *shock, horror* download the source and compile it. Chances are too you’ll get multiple errors, depending on if you are using 32-bit or 64-bit OS X.

I’m currently running Snow Leopard, and with a change to specify the path for my_config program and telling it to compile for 64-bit

Here’s the link that solved all of my problems :Mysqldb, python and os x: a match made in Satans bum

It’s still a manual process, but at least someone found out how to fix it.

Is facebook the new social control ?

Today I heard a funny thing that made me think of facebook and twitter and how it may become something that I thought had disappeared: a way of social control, of following and checking up on you.

We were in the Delaize. The cashier woman, age somewhere in her mid-thirties, while scanning our stuff, was talking to another person and was telling this person about somebody she knew; this person she knew had put his facebook status as ‘away to x the whole day’ but she had seen him and somebody else walking around, so they were both lying !

What struck me first was the casual way that Facebook was mentioned, as if it was an everyday tool to use. Funny how just a few years ago nobody in the Delhaize would have understood the word Facebook nor even what it meant. We’ve come far !

It also made me think of my days of youth, when living in a small village everyone knew everything about everyone. My mother could come home from shopping and for the next hour or so go into a spiel about what this person did to that person, who said what against someone else, all the blow-by-blow discussions and nasty detail. Names were given in a short/long version, like ‘marieke-van-achter-de-winkel’, ‘jef-with-the-hat’ that only insiders could understand.

When I grew up, I learned that this only happened in small communities, while in the big city you just sink into the anonymous mass.

However, social control may be making a comeback using new tools: while I use facebook to check on what my friends are doing (or not doing) it’s very easy to cross the line into the rabid following of everything your friends are doing online. More and more people are coming online, including your next-door neighbour.

This means that more and more people will know what you are doing, both those living close to you and those following you online. And I think a cross-pollination is occurring between the online world and offline world. If you tell a small white lie to a friend online that you are for example sick while you are not, chances are your neighbor or local friend who sees you working in the garden can point out the omission, while online, thus informing *everybody* what you were up to.

Soon the differences between online and offline will become vague, and the only line will be between those you know and those you don’t know.

Changing DEV from SQLite to MySQL

The last few weeks I had quite a few problems with SQLite – whenever I was updating the database using threads (so multiple connections at the same time), fairly frequently a database access error would show up.

Regular usage was fine, no problems, and before I had less than 15 keywords to scan which worked fine as well. Even so, there really shouldn’t have been a problem, as I used the same number of threads to access the database.

The problem might even lie in the interaction between Dropbox, which I use to store my development work and the sqlite database in there as well.

So to solve the problem and because at some point I do need to switch over, I set up a local MySQL database and connected to it. Works fine, and seems to be heaps faster to process the returned data.

So far so good, I don’t seem to have any connection problems any more. And the MySQL Workbench to analyze your schema and tables and user accesses is a joy to work with !

A few more things to do…

Just finished setting up this blog. Adding Akismet was the one big thing to absolutely do. Second thing was adding WordPress stats.

Now there’s only the rest of the setup todo.

  • Setting myself up as an independent
  • Finding an accountant
  • Finish writing my skeleton twitter search thingy
  • Putting it up on the server
  • Betatesting it until it no longer falls over and is a young and strutting program
  • Finetuning by iteration

Oh and fixing the blog image on the top, as it hangs out a bit on the right side. Since it’s of no consequence but the easiest todo, I’ll be doing that first, no ?