Monday 13 June 2011

The Earthquake that killed Twitter?

[For New Zealand visitors, the quick solution is probably to use geographical searching, although sometimes Twitter cannot keep up with the work that it needs to do: (eqnz near:"new zealand"). Or use a filter you'll have to keep updating.]

There was another earthquake in Christchurch today, strong enough that I felt long period shaking four hours' drive away in Dunedin, which made me very worried about my many friends up the coast. I did what everyone does in these situations, popped #eqnz into Twitter, and found myself connected to the stream of reports and rumours.

Initially I was a little concerned as most of the reports were from outside Christchurch, with few from the city itself, but this cleared up as networks, phones and power quickly came back to life. For about twenty minutes I, and the world, had a great source of news on the happenings, and a great sense of relief that, basically, everything was fine.

Then, just as we wondered why this thing wasn't trending, even in New Zealand, the #eqnz tag finally got noticed by Twitter and the inevitable happened.

Spam.

Rapidly created accounts with nonsense but early in the alphabet names flooded the tag, and those for popular movies and memes, with links to modestly disgusting porn and SEO sites. Fine, try to scam some movie or the latest tween obsession, but an Earthquake? Please.

Anyway, these people, we already know, have no compunction about their activities, and merrily waste bits and oxygen as they usually waste our time, but when they instead clobber a useful utility in a time of crisis, well, that utility is less useful.

And this is where this is Twitter's problem. Me, and a hoard of well-meaning Kiwis did our best to block these bloodsucking scum from the feed, but, and this amazes me, it takes four clicks to get from the search feed to finally reporting an account, and then, of course, it's too late anyway, as the spammer's gone and made ten or twenty more.

Twitter could do something useful here. First, they can fix the reporting for spam by adding a single big button to press, right there in with the search results.

Second, while people found they could get better news by using geographical filtering (eqnz near:"new zealand"), this still hides a lot of useful information. Twitter need to provide a way to search based on the reputation and activity of users, allowing and making it easy to only see tweets from the trusted, from accounts that have existed for more than three minutes, and which have activity profiles consistent with their belonging to real people.

If Twitter can fix that, they might survive, but if they cannot, the whole service is going down the drain...

Update: It gets much better once the tag isn't trending globally. I really hope twitter can sort something out for the next big distaster.

6 comments:

  1. The problem with the geographical filtering is that very, very few tweets sent about the subject of the earthquake are geotagged.. as far as I can tell that is what twitter uses for the 'near: filter'.

    For what its worth I set up this site http://chchneeds.org.nz immediately after the last big earthquake.. and it is still operating today. As a first step to finding tweets relevant to chch earthquake and filtering out spam it does also allow for filtering by address when people mention an actual address in the tweet.

    For example if they say 'at 23 Maidavale lane' or 'in Sydenham' it tags those tweets as being in a particular suburb and in the region of canterbury. For example:
    http://chchneeds.org.nz/#!/loc/canterbury

    I had grander ideas for this but it was all about people in canterbury choosing to actively use the tags #offer and #need. When I realised that eq.org.nz and their volunteers were doing a better job of filtering and sorting (and getting their message out) than i put my time into helping them.

    At this time eq.org.nz has been shut down as it does use quite a lot of volunteer hours to keep going, but everyone and anyone is welcome to try and use the data from chchneeds in anyway that they think it may helps. It's all open data made publically available of course. I'm @chchneeds on twitter if you want to get in touch with me

    ReplyDelete
  2. interesting that you can also discover the spam tweets by clicking on most popular tags co-associated with the #eqnz tag - basically automated bombing of popular hash tags it seems.. such a goddamn shame that a public resource like this is being ruined by these spammers.. you have a very good point

    ReplyDelete
  3. If there was an earthquake here I would be on http://bbc.co.uk/news because it has had 88 years of building up a reputation.

    When the iPhone has it built in they will find it turns to shite.

    ReplyDelete
  4. I have often thought it might be useful if you couldn't post to twitter for the first day after your account is created... that way you could get around miscreants who create an account to spam everyone at the time of a crisis/breaking news item etc. If the account has to exist for some time period then you are more likely to be able to block spammers I would think.

    Of course, the flaw I can see with this is in the context of govt oppression of speech in an oppressive regime or during a demonstration etc, where people might feel the need to create multiple accounts on the spot to hide their identity while reporting "bad things" going on. But really, since twitter are the only ppl that can take down twitter accounts this is probably not likely to happen in that short a time frame, and it's not like twitter will comply with takedown requests from oppressive regimes, (right?) so word would still get out...

    The spam issue might take a bit of work to solve, but shouldn't be insurmountable! But I do agree that Twitter is big enough and popular enough now that it's time to start tackling this issue with some priority...

    ReplyDelete
  5. This got popular on Hacker News, where there was some good discussion: http://news.ycombinator.com/item?id=2648305

    ReplyDelete
  6. One thing that was wrong was the idea that the spammers used retweets. Hardly any that I saw in the timeline were retweets at all. They did, however, use bit.ly and .us links, which were easier to filter out, although that meant legitmate information using bit.ly was scrubbed.

    I had to change to the modified tweet (MT) to relay anything.

    Yes, the geographic solution also made it harder for genuine emergency information relayers outside New Zealand to be helpful.

    I agree with the idea that new users shouldn't be able to tweet the first day or two of their account usage. That would have a huge impact on the spammer accounts and only genuine cases would have the patience to wait.

    ReplyDelete