Badges and Assessment (part 1 of 3)

In August Mark Surman wrote an enticing post entitled Experiment: Badges, identity and you (http://commonspace.wordpress.com/2010/08/12/badges-identity-and-you/). Fast forward a month and in comes me, Joshua Gay (hello!). I've come on board to work on two projects: to help develop the badge backpack as well as to work with P2PU.org to experiment with peer-based educational assessment. The relationship between the two is simple: some peer-based assessments on P2PU will result in students being given digital badges; these badges will then get stored in a digital ''backpack''. 

Seems simple enough, right? Well, that's what I thought as well, until I started really looking into each of these ideas. What I discovered is that there are a great number of tantalizing questions, exciting implications, and possible implemenations of badges, a badge backpack, and of peer-based assessment models. So, as a sort of status report to the community, I've put together a series of three posts, of which this is the first. 

Badges, badges, everywhere

Once you start to think about badges in both your digital and analog lives, you realize that they turn up all over the place. In the physical world some examples of where badges are awarded are:


Perhaps my favorite example of a badge system that I've come across is the  Peace Corps Merit Badges (http://www.peacecorpsmeritbadges.com/). You can read about the badge below in the story, Peace Corps OLPC Merit Badge: Hardest Badge You'll Ever Earn (http://www.olpcnews.com/people/volunteers/peace_corps_olpc_merit_badge_h.html ). 

IMAGE: http://www.olpcnews.com/images/olpc-badge.jpg

In the digital world, badges can be found across all sorts of sites and networks, some examples are: 


One of my goals in exploring badges across these wildly different contexts is to understand what the commonalities are amongst badges. Our goal of course is to construct a general placeholder that any organization could use to describe their badge. 

Badge Semantics

<blockquote>
Shrek: Ogres are like onions.
Donkey: They stink?
Shrek: Yes. No.
Donkey: Oh, they make you cry.
Shrek: No.
Donkey: Oh, you leave em out in the sun, they get all brown, start sproutin' little white hairs.
Shrek: NO. Layers. Onions have layers. Ogres have layers. Onions have layers. You get it? We both have layers. [sighs]
Donkey: Oh, you both have layers. Oh. You know, not everybody like onions. 
</blockquote>

Like Ogres, onions (and parfait), badges, too, have many layers. 

The first layer I have unravelled are basically the five W's of a badge, which are: 


These five properties seem to be common across most badges. But, as you would expect, badges often get slightly more complex and can have more layers of meaning. Some of the ones I've come across are: 

Finally, the other layer of badges I have discovered have to do with the relative meaning of badges. For example: 


Badge Syntax

As we begin to sort through the various semantic properties we might use to characterize any given badge, we are also looking toward different ways to represent our badges syntactically. A few of open data formats that have come up in discussion thusfar include microformats, RDF/RDFa, and ATOM. 

Ultimately our goal is to make it easy for other projects and individuals to create, award, share, and display badges. The biggest challenge we face in representing badges is not going to be representing the basic five W's, but, rather, the meaning of the badge as it relates to the context in which it was granted as well as the meaning of the badges relative to one another. 

Some of the leading questions we have come-up with are: 


The context we are asking ourselves these questions is that we want to think about how projects can make use of badges. How can they be used? How, why, when, and to whom will you display your badges? 

Next Steps

So, now that I have you thinking about badges, let me know what you think. I'd love to hear your thoughts on badges; on ideas of how to represent badges; and to get any other questions you might have. 

In our next segment we will be exploring badges further in the context of educational assessment and P2PU.org. 


********************************************************************************************

Badges and Assessment (part 2 of 3)

In our first segment we explored the general idea of badges. We discussed various examples of where they appeard in the physical and virtual world. Then we attempted to begin listing the various semantic properties badges have as well as briefly touching upon thoughts on the syntactic representation of badges. 

In this segment, we will explore badges in the context of our experiments in educational assessment on P2PU.org. 

OUTLINE OF POST


Motivation; assessment OVERVIEW, IDEAS, MOTIVATIONS


Badges  and assessment


Assessment: the big picture; speculation; call for discussion 

********************************************************************************************
Badges and Assessment (part 2 of 3)

Imagining a Badge Ecology


Next Steps: 

How to get involved: 

*******************************************************************************
NOTES

Compare  .deb package management systems: 



badger badger badger: http://www.badgerbadgerbadger.com/


Microformats 
microformats are a set of simple, open data formats built upon existing and widely adopted standards.
http://microformats.org/- 

The list of current, stable Microformats open standard specifications. 

RDF & RDFa

RDF is an XML format. 

*******************************************************************************************


The P2PU Assessment Experiment
    
  The Peer 2 Peer University (P2PU) is an online community of open study  groups for short university-level courses. Think of it as online book  clubs for open educational resources. The P2PU helps you navigate the  wealth of open education materials that are out there, creates small  groups of motivated learners, and supports the design and facilitation  of courses. Students and tutors get recognition for their work, and we  are building pathways to formal credit as well.

But, P2PU can also be thought of as a experimentation platform. This is  because we are breaking new ground and will be constantly building and  improving our system to adapt to the needs of the using community. To  figure out what works, and what doesn't work, we will need to experiment  as we go. 

One of the current experiments we are working on is that of educational  assessment models and recognitions that can be mapped onto the P2PU  platform. Another experiment we have going is the use of badges on the  P2PU system. These two experiments intersect when a badge is rewarded as  an outcome of a particular assessment taking place. 

To give a concrete example, imagine you have identified the skill of  being able to give helpful responses in discussion threads. One way we  could imagine assessing this skill is to have peers rank or critique  student responses within discussion threads. With a little imagination,  you could imagine that if a student has received enough positive  responses of a certain nature and in a certain context that this could  indicate that it is likely a student has achieved some level of  competency at the skill were were trying to assess. As a result, that  student is rewarded with the "helpful responder badge." 

This experiment itself is interesting and it is certainly one we will be  pursuing. But, let's imagine we bring it a step further. Imagine if  once you have received the "helpful responder badge", you suddenly find  out that you have unlocked other assessment models and abilities of  acquiring new badges you didn't' have before. 

At this point, you may be thinking that things are starting to get  really exciting, or perhaps you are thinking they are starting to get  really scary! In either case, these kinds of creative and interesting  use of badges and assessment leave us with many exciting possibilities  and experiments to conduct. 

Right now, our goal is to begin awarding badges across some of our courses that are part of the School of Webcraft.  But, the outstanding question is how and when will we award these  badges? If we award them, will they be helping us to authenticate  learning outcomes and if so, should we also use badges for other reasons  such as to motivate student engagement or as a sort of administrative  filtering tool? 

When I think of many and varied uses of Badges, one of the first thing that comes to mind is the US Military. 


US Military Badges


In the US Military their are all sorts of ribbons, badges,  and medals. The meaning of each is extremely context sensitive. Some  badges or ribbons are relevant only to a particular Unit, some indicate  skills, achievement, merit, or are awarded simply because you happened  to do something (like go overseas). And yet some go across all  boundaries, for example, members across all branches of the uniformed  services are encouraged to render salutes to recipients of the Medal of  Honor as a matter of respect and courtesy regardless of rank or status. 

For example, look at variety and complexity of accoutrement on this sailor's uniform: 


IMAGE: http://upload.wikimedia.org/wikipedia/en/9/97/Navy_Ribbons_and_Badges.jpg


The P2PU Take-away
I think that as we begin to figure out how to integrate badges  into the P2PU system we can learn a lot about the sophisticated use of  these various accoutrement in the US Military. 

In particular, I think that we can have different uses of badges across  the system, even if at times the use of badges might be for totally  different reasons (some of which may even seem in opposition with one  another). But, in order to pull it off, special attention will be need  to be paid to three different aspects of badges.
 
  Visual display of badges.  The size, complexity of design, and the relative layout of badges are  all going to contribute to how people interpret and understand them. I  think if we make visually distinct types of badges and allow for them to  be grouped in different ways then people will be able to immediately  tell when a badge relates to their serious learning goals or when it is  simply being used as a filter or hoop to ensure a person is not a bot or  is actually going to commit to something. 
  
  Context, label, and process of being awarded a badge. If you are going along the system and all of a sudden a box slides out and states, "You have earned the helpful responder ribbon  because five people have found your comments to be helpful; you now  have the ability to flag some posts as SPAM" and then the box slides  back down, you might smile and go on with your day. But, on the other  hand, if you received a full page notice and email explaining why and  how you were awarded the "Community Leader Merit Badge."  This might give you pause and encourage you to reflect upon both your  personal learning goals and your past and present experiences in your  course of learning. 

Relative meaning of badges. As  soon as a sufficiently large number of badges begin to be created both  on our system and on other systems their will be an increased need for  people to begin comparing badges in various ways. If we know this going  into it we can encourage people to create comparison grids or to  organize their badges in ways that make clear how and when a badge is  rewarded and what the (as a programmer might say) the various  preconditions and postconditions are for a given badge. 


But Does This Actually Work?

Of course, it is easy to go through this kind of a deconstruction  exercises on military badges and then simply attempt to apply the  salient principles you draw from this and apply them to P2PU in the  abstract. But, the question remains, will badges still work as we try to  achieve our goals of building an assessment system in P2PU? 

I think that in order to answer this question we need to ask more refined questions. For example, we could ask, will  the more "game-like" uses of badges that simply unlock functionality  for users interfere with badges that are associated with formative  assessment and recognition?
  
The answer is that we don't know. But, that's OK. Coming-up with  these kinds of refined questions are essential for us to be able to  conduct effective and meaningful experiments through the P2PU platform.

Over the next few weeks we will be awarding a few badges in our School  of Webcraft courses. As we do so, this will hopefully give us a first  opportunity to both put this question to the test as well as to begin  experimenting with how we display and present badges to users. 

This experimentation will continue through the fall and winter and as we  begin to build more sophistication into our Badge system and our P2PU  platform so too will we be able to perform more sophisticated  experiments in this area. 

Lastly, we are not doing this alone. We are also hoping to work closely  with groups such as Remix Learning and MAKE Magazine who are also  interested in offering badges on their systems and to their  communities.  The coming months should be exciting and we welcome you  will join us in all the fun!