Crowdsourcing journalism: The Guardian way

From the Nieman Journalism Lab by Michael Andersen:

Four crowdsourcing lessons from the Guardian’s (spectacular) expenses-scandal experiment

Okay, question time: Imagine you’re a major national newspaper whose crosstown archrival has somehow obtained two million pages of explosive documents that outed your country’s biggest political scandal of the decade. They’ve had a team of professional journalists on the job for a month, slamming out a string of blockbuster stories as they find them in their huge stack of secrets.

How do you catch up?

If you’re the Guardian of London, you wait for the associated public-records dump, shovel it all on your Web site next to a simple feedback interface and enlist more than 20,000 volunteers to help you find the needles in the haystack.

Your cost for the operation? One full week from a software developer, a few days’ help from others in his department, and £50 to rent temporary servers.

Journalism has been crowdsourced before, but it’s the scale of the Guardian’s project — 170,000 documents reviewed in the first 80 hours, thanks to a visitor participation rate of 56 percent — that’s breathtaking. We wanted the details, so I rang up the developer, Simon Willison, for his tips about deadline-driven software, the future of public records requests, and how a well-placed mugshot can make a blacked-out PDF feel like a detective story.


He actually offered SIX lessons. Here they are in a gist:

1. MAKE IT FUN. Willison lured the readers by making it feel like a game. The Guardian’s four-panel interface — “interesting,” “not interesting,” “interesting but known,” and “investigate this!” made categorization easy. And the progress bar on the project’s front page, immediately giving the community a goal to share. He added the Guardian’s mugshots of each MP to their pages in the database, which gave a personal element. “You’ve got this big smiling face looking at you while you’re digging through their expenses.”

2. MAKE IT COMPETITIVE. Willison posted lists of the top-performing volunteers. “Any time that you’re trying to get people to give you stuff, to do stuff for you, the most important thing is that people know that what they’re doing is having an effect. It’s kind of a fundamental tenet of social software. … If you’re not giving people the ‘I rock’ vibe, you’re not getting people to stick around.”

3. LAUNCH IMMEDIATELY. Before Parliament released its records Thursday, Willison’s team thought they might be able to postpone their launch to Friday if necessary. When they saw Thursday’s newsbroadcasts, they realized they’d been wrong. The country’s imagination was caught. “It became quickly clear on Thursday that it was a huge story, and if we failed to get it out on Thursday, we’d lose a lot of momentum.”

4. USE A FRAMEWORK. Willison’s project was built on Django, the custom Web framework “for perfectionists with deadlines” that he and Adrian Holovaty created for the Lawrence Journal-World. Other frameworks and languages would have worked, too. “You absolutely could build this in Ruby on Rails or in PHP,” Willison said, but “as far as I’m concerned, this is absolutely Django’s sweet spot. This is absolutely what Django is designed to do. Once I had a designer and a client-side engineer working on the project, I could really just hand it over to them and I didn’t have to worry about the front-end code any more.”

5. HAVE SERVERS READY. As well as the Guardian’s first Django joint, this was its first project with EC2, the Amazon contract-hosting service beloved by startups for its low capital costs. Willison’s team knew they would get a huge burst of attention followed by a long, fading tail, so it wouldn’t make sense to prepare the Guardian’s own servers for the task. In any case, there wasn’t time. With EC2, the Guardian could order server time as needed, rapidly scaling it up for the launch date and down again afterward. Thanks to EC2, Willison guessed the Guardian’s full out-of-pocket cost for the whole project will be around £50.

6. SAVE COSTS. Willison used open-source, freely available software that anyone else who might want to imitate them could use.

MORE.

Twitter rules for journalists

Julie Posetti has posted a set of guidelines for journalists when using Twitter.

You will find some of these obvious and others contradictory and meaningless (be human, be honest but don’t bitch about your workplace), but kudos for trying to frame Twitter in a context that some journalists can understand — or choose to ignore.

Top 20 Take Away Tips for Tweeting Journos

1) Think before you tweet — you can’t delete an indiscreet tweet! (Well, you can, but it will survive in Twitter search for three months and it’s likely live on as cached copy somewhere.)
2) Think carefully about what you’re re-tweeting and acknowledge if it’s unsubstantiated.
3) Be an active twit: tweet daily if you want your followers to stick.
4) Determine your Twitter identity.
5) Be human; be honest; be open; be active.
6) Don’t lock your account if you want to use Twitter for reporting purposes — this fosters distrust.
7) Twitter is a community, not just a one-way conversation or broadcast channel — actively engage.
8) Check if your employer has a social media policy.
9) Be cautious when tweeting about your employer/workplace/colleagues.
10) Be a judicious follower — don’t be stingy but avoid following everyone as your list grows to avoid tweet bombardment.
11) If you quote a tweet, attribute it.
12) Expect your competitors to steal your leads if you tweet about them.
13) Don’t tweet while angry or drunk.
14) Avoid racist, sexist, bigoted and otherwise offensive tweets and never abuse a follower.
15) Scrutinize crowdsourced stories closely.
16) Find people to follow. Foster followers by pilfering the lists of other twits.
17) Twitter is a ‘time vampire’ (via @anne_brand) — you don’t need to keep track of all tweets, so dip in and out through the day.
18) Prevent information overload by using an application such as Tweetdeck.
19) Add applications to your Internet-enabled mobile device to allow live-tweeting on the road.
20) Add value to your tweets with links, Twitpic and other applications for audio and video.

More.

Wall Street Journal’s silly rules on social media

[via Editor & Publisher]

Just to show you how far removed and laughable the supposedly “smartest people in the room on journalism” are, here are the recent rules introduced by WSJ for its staff’s social media activities:

* Don’t recruit friends or family to promote or defend your work.
* Consult your editor before “connecting” to or “friending” any reporting contacts who may need to be treated as confidential sources. Openly “friending” sources is akin to publicly publishing your Rolodex.
* Let our coverage speak for itself, and don’t detail how an article was reported, written or edited.
* Don’t discuss articles that haven’t been published, meetings you’ve attended or plan to attend with staff or sources, or interviews that you’ve conducted.
* Don’t disparage the work of colleagues or competitors or aggressively promote your coverage.
* Don’t engage in any impolite dialogue with those who may challenge your work — no matter how rude or provocative they may seem.
* Avoid giving highly-tailored, specific advice to any individual on Dow Jones sites. Phrases such as “Travel agents are saying the best deals are X and Y…” are acceptable while counseling a reader “You should choose X…” is not. Giving generalized advice is the best approach.
* All postings on Dow Jones sites that may be controversial or that deal with sensitive subjects need to be cleared with your editor before posting.
* Business and pleasure should not be mixed on services like Twitter. Common sense should prevail, but if you are in doubt about the appropriateness of a Tweet or posting, discuss it with your editor before sending.

MORE.

The Future of Journalism: Emily Bell

Emily Bell‘s speech of the Future of Journalism is worth reading in its entirety or get the gist from this blog.

Some highlights:

Recently Clay (Shirky) kicked off a terrible noisy feedback loop of chatter about the future of journalism when he talked about it in the context of the introduction of the printing press and pointed out that everyone talked about the revolution without acknowledging what happens in the wake of great revolutions and how this was informing the collapsing nature of our business.

This is his quote:

“That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen.”

The old stuff has broken before the new stuff is ready. Old stuff is certainly broken, many of our old media brands: ITV, The Independent, The Scotsman, Five Channel 4….in fact, outside the very deep pockets of News Corporation, the family commitment of Associated Newspapers, the unusual funding structures of The BBC -and the Guardian for that matter – just about everywhere the economy and broadband delivery are breaking bits of the media at an alarming pace….

Just to really cheer you up so far this year, and remember companies also cut jobs last year, 900 jobs have gone from newspaper groups, a similar amount from TV stations, a couple of hundred from magazines and radio – but this doesn’t include the many more casual contracts that have been terminated or the contributor budgets that have been cut.

All of this is very sad, but much of it is inevitable, and, more than that, once the pain has gone away and countless titles and brands have closed, which I still believe they will, there will be a new order of journalists and organisations, many of them shaping a future in a way which it is difficult for pre-revolutionary businesses to even imagine and whilst it might seem unsympathetic to say so we will look back in a few years time and wonder at why it took us so long to change.

We can carry on describing the problems journalism and news organisations face until the cows come home, or indeed are shipped off for slaughter in the wake of foot and mouth…

If there is not a cast iron solution, already, still in the depth of recession there are many clues, and clues are useful because cluelessness is one of the media’s key problems at the moment.

CLUE ONE – any communication organisation needs an audience. So find one. If you build it they won’t come because they are busy elsewhere. So go where the audiences is.

The idea that we can shepherd viewers or readers or listeners into one place at one time is gone – we all consume our news comment and analysis through many, many different sources.

If your audience is declining as it is for primetime analogue tv, newspapers, and radio stations, but reading, watching, listening is growing, as it really is, then go where the growth is, be on all platforms. Use Twitter, Facebook, mobile, youtube, podcasts, email and sms. Don’t be afraid to let your content go – what’s the worst that could happen.

Do not just do more of the same – Einstein’s definition of insanity was to do the same thing over and over again expecting different results.

CLUE TWO – networks work better than silos. Rules of networks are that if you are a hub not a destination your traffic will be higher….the same is true of media outlets, the same is true of individual journalists or stories – if you are at the centre of your community not on the periphery, many people will go through you.

CLUE THREE – utility and reliability never go out of fashion and trustworthiness and transparency are crucial. When people know you will tell them something useful they will seek you out, when they know you are trustworthy they will tell you things so you can tell others. And journalism means journalists – trust is placed in people as well as brands.

Robert Peston, the BBC’s business editor is an exponent of this – he is a tall poppy who has gathered a following which gives him 1m hits on every blog post – people want to find the good stuff and to have a personal relationship with a trusted source.

The internet allows that. But Robert Peston has a following because he is very well informed by his sources and can tell you things which, on the whole, other people can’t. He is a classic example of where the BBC, the country’s largest journalistic employer, is moving from a bulletin led model to a correspondent led model. They pay Peston to spend time understanding complex stories and cultivating sources… This is basic stuff but oh so important, and there can be no better use of a news organisation’s resource than allowing its journalists that space and time.

CLUE FOUR – Wikipedia is often a better historical source on news stories than news organisations themselves. There are two lessons here – one is that the news business is struggling to understand the language of the web, the second is that tools plus users equals content, both are key to the future of journalism.

Matt Waite is 32 and he has just won a Pulitzer Prize for his Politifact website for the St Petersburg Times in the US. It checks facts around what people in Washington put in their speeches. Matt is now a news technologist, but WAS an investigative reporter – he’s a brilliant example of how you can reinvent a strand of journalism in line with the way the web works..The site is based on DATA – and allows the user to study that data – it tells a story, not in the conventional way, but in a way more powerful on the web.

CLUE FIVE – not really a clue more a statement of the obvious, as Dan Gilmour, the famous media journalist said in his seminal book “we the media” “there’s always someone closer to the story than you”, or as my mum said, at the kitchen table, “whenever I read or see something I know anything about I’m always struck by how wrong it is”.

It amounts to the same thing – your readers and audience know and see more than you ever could. Find ways to let them add their knowledge.

MORE.

Can design save newspapers?

Jacek Utko at TED:

Shooting Shirky

After reading Shirky today, I can see editors lining up with their shotguns and shouting in unison — “Pull!”.

Again Clay, in his usual cerebral way, has taken the complex and reduced it to simple cold, hard facts, weaving in his wikipedian understanding of the Internet and making news owners seem like grumpy old men stuck in the mud and missing targets.

Sometimes he loses me — perhaps this simple mind can’t seem to grasp at the core of his message — but thank god for the webbiness of random access and infinite repetition and Google and YouTube. I can read and watch him again and again to finally Get It.

To appreciate the genius of Shirky, you need to step back to his 2005 TED talk on Institutions vs Collaboration.

Then listen to his entertaining “cognitive heatsink” theory at the Web 2.0 Expo: Part 1 | Part 2

Quote:

This is something that people in the media world don’t understand. Media in the 20th century was run as a single race–consumption. How much can we produce? How much can you consume? Can we produce more and you’ll consume more? And the answer to that question has generally been yes. But media is actually a triathlon, it ‘s three different events. People like to consume, but they also like to produce, and they like to share.

And what’s astonished people who were committed to the structure of the previous society, prior to trying to take this surplus and do something interesting, is that they’re discovering that when you offer people the opportunity to produce and to share, they’ll take you up on that offer. It doesn’t mean that we’ll never sit around mindlessly watching Scrubs on the couch. It just means we’ll do it less.

And this is the other thing about the size of the cognitive surplus we’re talking about. It’s so large that even a small change could have huge ramifications. Let’s say that everything stays 99 percent the same, that people watch 99 percent as much television as they used to, but 1 percent of that is carved out for producing and for sharing. The Internet-connected population watches roughly a trillion hours of TV a year. That’s about five times the size of the annual U.S. consumption. One per cent of that is 100 Wikipedia projects per year worth of participation.

I think that’s going to be a big deal. Don’t you?


Then go back and re-read Shirky firing away at the clay pigeons falling from the sky:

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away.

The curious thing about the various plans hatched in the ’90s is that they were, at base, all the same plan: “Here’s how we’re going to preserve the old forms of organization in a world of cheap perfect copies!” The details differed, but the core assumption behind all imagined outcomes (save the unthinkable one) was that the organizational form of the newspaper, as a general-purpose vehicle for publishing a variety of news and opinion, was basically sound, and only needed a digital facelift. As a result, the conversation has degenerated into the enthusiastic grasping at straws, pursued by skeptical responses.

“The Wall Street Journal has a paywall, so we can too!”(Financial information is one of the few kinds of information whose recipients don’t want to share.)
“Micropayments work for iTunes, so they will work for us!” (Micropayments only work where the provider can avoid competitive business models.)
“The New York Times should charge for content!” (They’ve tried, with QPass and later TimesSelect.)
“Cook’s Illustrated and Consumer Reports are doing fine on subscriptions!” (Those publications forgo ad revenues; users are paying not just for content but for unimpeachability.)
“We’ll form a cartel!” (…and hand a competitive advantage to every ad-supported media firm in the world.)

Round and round this goes, with the people committed to saving newspapers demanding to know “If the old model is broken, what will work in its place?” To which the answer is: Nothing. Nothing will work. There is no general model for newspapers to replace the one the internet just broke.

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. It makes increasingly less sense even to talk about a publishing industry, because the core problem publishing solves — the incredible difficulty, complexity, and expense of making something available to the public — has stopped being a problem…

…When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

More.

The news reporter of the future is here

The newsman of the future is already here. And he isn’t some geeky, know-it-all 20 year-old. Steve Rubel of Micro Persuasion says he’s Peter Abraham, the Yankees beat writer for the Journal News in Westchester county (circ. 100,000), NYC, the 94th largest newspaper in the US.

Quote:

Abraham is on the scene in Tampa where the Yankees are training and he’s doing it all – in addition to filing regular reports for the paper that appear in print. Here’s an inventory of his social media footprint….

First, he has a blog with a full-text feed that includes several posts/day and hundreds of comments/day from readers. It dates back to 2006.

In addition, Abraham has a Facebook group that has about 1600 members.

He is posting photos from spring training using his iPhone. Note the gear the others are using by comparison.

There is a podcast up on iTunes that right now is updated daily with audio.

Finally, today he was using both CoverItLive and Mogulus to have a live video/text chat with readers.

All Abraham is missing is Twitter, YouTube and maybe Flickr but he seems to be doing just fine with what he has here.

In the words of the Pitbull of Personal Development “Shut up, stop whining and get going already”.
MORE.

What journalists want in your website

In my previous guise as an online journalist, I used to be very frustrated with corporate websites that lacked the immediate background info needed for my stories.

Very often they didn’t even have a Newsroom or Media section or the UI or search was so clunky it was impossible to find stuff like when the CEO was named the CEO, the company’s headquarters or specifics on when a product was launched.

Thank god, Google came along and made search within sites easy with the operator site:abc.com.

In training, we emphasise the need to have a Quick Facts or FAQ section specifically for the media. When archiving, make press releases searchable by quoted exec, topic, brands, product, service and date can help ease the journo’s job.

On the other end, we teach simple Google tricks which go beyond just typing what you want in the search box.

Useability guru Jakob Nielsen interviewed 40 journalists to get their views. His conclusions are a worthy read:

Journalists are not gullible, and they don’t take a company’s own word as truth. Indeed, almost all journalists said that press releases were useful only to find out how a company is trying to position itself. We strongly recommend that PR areas have links to external sources, including press coverage; journalists often consider articles from independent newspapers and magazines to be much more credible than a company’s own press releases. We’ve seen similar findings in studies of prospective customers evaluating products on consumer- and business-oriented sites, so links to external press coverage can also help promote sales.

The top-5 reasons journalists gave for visiting a company’s website are:

* Locate a PR contact (name and telephone number)
* Find basic facts about the company (spelling of an executive’s name, his/her age, HQ location, and so on)
* Discern the company’s spin on events
* Check financial information
* Download images to use as illustrations in stories

This basic information must be easy to find and should be cleansed of the marketese and excessive verbiage that smother the facts on many sites. Journalists don’t have time to wade through deep, complex navigation trees or sift factual wheat from marketing chaff. In particular, pages must present information in well-organized chunks that are easy to scan. Distracting animations and irrelevant stock photography don’t help journalists who are in a hurry to find the facts.

MORE.

Citizen Journalism 101

This Flash explanation of Citizen Journalism is brilliant, despite the typos. From VizEdu.com:

The dawn of the “personal journalist”

WANTED: Personal journalist. Must be able to gobble up massive quantities of media – RSS feeds, blogposts, tweets, Facebook updates, podcasts, Youtube videos, Flickr photos on daily basis – and turn it into easily digestible bits of gold for superbusy C-level executive.

Michael Hedges suggests the possibility of the personal journalist – similar to that of the personal trainer or personal shopper:

The personal journalist is a wave or two beyond the tried and busted ‘user-generated content’ and ‘citizen journalist.’ Neither found a business model. Citizen journalism, a term invented by accountants, past its prime when listeners, viewers and readers lost interest in ‘reports’ from the 16 year old on the corner with a cell-phone camera. Blogs, touted as giving voice to many, became, largely, ranters ranting to themselves or PR people posting the daily spin. Blog creation has peaked, wrote the Pew Research Center in a 2007 report. The successful became niche publishers, albeit of the traditional media model. The rest are just out there, hanging by the Web.

User-generated content is another concept designed to warm the accountants’ books. Couple it with the much vaunted social networking sites and zillions of web hits are created. All content may, indeed, be equal for 20 year old user/creators but an adult looking for knowledge and clarity is left empty. Unfortunately, sources for adults have evaporated into the dither of click-through ads.

Imagine a typical appointment with the personal journalist. Busy humans, executives in their own right, need to acquire knowledge efficiently. While an executive might be buried in reports and analysis and numbed by 15 news channels, the personal journalist answers the need by asking the right questions, getting the story and delivering it in 750 words, voice or text, to the device of choice. Is it worth €75 a hour – the rate personal trainers charge – to avoid the 3 hours sifting through 250,000 search engine results, only 2 of which might yield a shred of knowledge?

The personal journalist does, in fact, do what journalists do best: keep their eyes open. In addition to providing the news you need, the personal journalist brings serendipity to the day. What a joy it would be to have that concise ‘news for you’ mixed with the occasional ‘…and think about this.’ And best of all, the personal journalist is ‘on and gone.’ No promos. No pop-up ads.

Media in the 21st century is buckling under the empowerment of users to get the information and entertainment they want, whenever and however they want it. The next step in that empowerment provides both clarity and expertise with greater sensitivity to time economy. Sad for some, technology will take a smaller role.

Of course, you might want to share your personal journalist with others. What might that be called?

MORE.

← Previous PageNext Page →

Feedburner RSS
Subscribe by RSS
RSS logo
Subscribe by email

Facebook TrinetizenTwitter TrinetizenLinkedin Trinetizen