measurement

{learning} The Art of Evaluation_#ICM820

learning

We at #ICM820 course have discussed successful strategies of community-building as well as cases gone very wrong. But evaluating – quantitatively measuring and qualitatively assessing – successes is a tricky issue.

From a macro-level vantage point of societies and its institutions, we could ponder how to assess media systems (or, as many tend to say about the digital era of multiplicity, media ecosystems) work effectively, democratically, openly, and so on. I have collected some links to projects, ideas, and cases that aim at measuring media systems and media development from a global perspective.

The meso-level of organizations outlook would be to look at effectiveness of particular political, economic, social, etc. communities, organizations, or campaigns. Is it about eye-balls, likes/shares/follows, comments, retweets/repins etc.? Is it about the ratio between lurkers vs. active participants? Professionally, do we value media differently than we did before, in terms of it as an advertising distribution tool, a news source, a forum for debate, an entertainment source? Here are just some examples of the infinite amount of views on how to measure success and impact in the digital age:

Also:

Finally, at the micro – or individual – level: How do you (does one) measure a digital community? Usability, access, relevance, engagement/familiarity, security…?

As experts of digital communities, how do we balance structural/technological concerns, big data metrics, and individual experiences?

{research} Assessing Media Freedom Indices

Research

The DW Akademie has just published a practical report on how to understand different ways to measure media freedom. Having been a part of the global Mapping Digital Media project (MDM, 2009-13) by the Open Society Foundations, I have become extremely interested in what we really need to know in order to assess how media and functioning (democratic) societies are linked to one another.

What the DW research highlights (and what was ‘done differently’ in MDM as the research teams were all local), is the ‘Western-Centric’ approach of these measurements:

In the analysis, media researcher Laura Schneider uses data as well as interviews with experts to examine the methodology used in each of the five best known press freedom indices. She takes a closer look at the organizations doing the ranking as well as how they conduct their respective ratings. Schneider also investigates the advantages and disadvantages of each index and points to aspects which could be improved.

Schneider’s biggest criticism is that the ratings aren’t objective, often being carried out by a handful of academics or media experts who are mostly from western countries. This inevitably results in a bias, especially as the majority of the ranking organizations fail to state their exact definition of media freedom.

“Each ranking has particular strong points and weak points and it’s important to know what these are, especially when using them to determine international media policies and the distribution of aid,” Schneider says.

 

Read the full report here: Media Freedom Indices.