BADGE INTEGRATION IN  LERNANTA (0.1) -- For more background and the old text of this pad /  future things to add after the pilot, see time slider

Todo list:

• Send   email to Erin, John, Jessica, Philipp, Arlton, Chloe so they can give  feedback before the end of august of the time for which jessica will be   doing the last bits of work on PD on P2PU metrics and starting   implementing badges integration.
• Mockups and/or  questions from Jessica to figure out what was not clear from what we will implement for Web Making 101

Vocabulary:

• Badge: what is awarded as recognision for learning or activity on p2pu.org.
• Bagde Challenges: what users have to accomplish or complete to earn a badge.
• Assesment Logic: different customizable logics for how badge challenges can be completed and the associated badges awarded.
• Tasks: where the peer learning takes place inside a study group or course.

Objective:
Building basic assessment functionality into Lernanta for the School of Webcraft's WebMaking 101 course. This includes:

• support creation of badge challenges
• promote/display badges, badge challenges, and awarded badges within the P2PU platform
• allow users to send the badges they earn to the Open Badge Infrastructure (OBI).
• track and report metrics

Core Features/Functionality:

ROLES

(1) Bagde Admins (new role on p2pu)

• To  allow schools and p2pu to endorse awarded badges (and provide challenge  ideas to be used by multiple groups/courses), some or all the initial  badge challenges will be created by people appointed by the schools or  p2pu.
• These people will need the necessary UI to create and manage challenges and the badges associated with them.
• Creating a badge challenge includes selection and customization of the assesment logic and the badge that will be awarded.
• We  will allow superusers and school organizers to select badge challenge  creators for p2pu or for the schools (note that superusers and school  organizers should not be disallowed to do what badge admins do).
• Badge  challenges created badge admins of a school will be associated to the  school as well as the corresponding badges (in general the badge  challenge will display information about who -- p2pu, a school, or a  group/course -- endorse them).
• In  terms of priorities it is more important to allow schools (SoW in  particular) to choose their badge admins, since the WebMaking 101 is  part of a school. Keep in mind that the rest will come in the future but  focus on the minimum functionality.

(2) Study Groups/Course Organizers (existing role)

• Depending  on the assesment logic behing them, some badge challenges or all of  them will require the interaction between users (e.g. user submits some  piece of work for review as proof that (s)he deserves the badge, and  someone has to review it).
• This interaction can take place inside a study group or course (this is the case of the web making 101 course).
• We  need to confirm if the mentors functionality that will be included at  the same time will have a role in the assesments logic (todo: ask John)  and thus there will be some interaction occuring outside of groups and  courses.
• In  order to bring badge challenges into the courses, organizers (maybe  participant too if it is not a course but a study group) will select  badge challenges endorsed by their school or p2pu to be displayed as  tasks inside their study groups or courses.
• It  is possible that the Web  Making 101 course will  require a few badges  not endorsed by the school, to give  a sense of  completion to other  tasks of their course, or to the actual  course. If this happens then  there will be badge challenges and badges not created by badge admins  and associated to specific groups or courses. (todo: ask John)
• The  developer(s) implementing the functionality for integrating badges into  lernanta will make the desicion of how much of the necessary logic will  be implemented for all tasks   and which part only for the badge  challenges.
• After  asking John we will get a better idea, but from what we know at the  moment the first priority here is to allow selecting endorsed badges to  be part of the task list of a group or study group.

(3) Study Group/Course Participants (or users that want to get a badge)

• Some badges will be awarded to them automatically (based on metrics tracked in p2pu).
• Others  will require users to submit pieces of work to a badge challenge, and  review eachother work. Kind of what happens currently with comments in a  task but now with the clear objective of awarding a badge.

ASSESSMENTS LOGIC

• Stealth assessment:
  • This kind of assessment logic will allow us to automatically award badges based on metrics collected inside p2pu
  • We  have to see how much customization we can put into them (i.e. if we  can/want rely to a certain degree on superusers, badge admins and group  organizers for their customization and/creation) but as a first step we  can define them at the source code level and let those creating the  badge challenge to choose from the list of stealth assesments we  provide.
  • we  will need help selecting names, images and logic for them, but to speed  things up or we can implement examples based on the metrics we are  collecting and ask for feedback.
  • examples  of things we could give recognition for with stealth assesments are:  visited course pages X numbers of days in a week, posted X numbers of  comments, reviewed X numbers of submitions.
  • on  the first badge pilot we did not add badges that were awarded based on  stealth assessment but the software that we customized for the pilot had  badges like these to encourage certain behaivours among their users.
  • In  terms of priorities stealth assesments are first priority and which  ones we will support first will depend on the tracking/metrics we are  implementing for PD on P2PU
• Peer assessment:
  • This kind of assessment logic will allow users to get recognision from their peers
  • In  some cases it will allow users to submit their work for review (e.g. to  receive recognition when they learned something) and in others users  will nominate their peers directly (e.g. to give recognition to their  peers for qualities like good team player)
  • Both cases have the roles of "candidate for earning a badge", and "peer evaluating if the candidate should earn the badge"
  • The logic behind them will vary depending on:
    • the  scale defined for votes and and threadholds of votes and number of  peers evaluations to issue the badge (e.g. we can have peers evaluate  from a scale from 1 to five and require a 3 or higher rating from 3  assessors to issue the badge)
    • if a rubric consisting of yes or no questions (checkboxs) is provided to help/automate a bit how rating is given.
    • if a submition is necessary from the candidate that will earn the badge
    • what wll be evidence that will be associated to the awarded badge once the challenge is completed by a user.
    • if it has to check that the candidate already have some prerequisit badges.
  • The  first badge pilot showed that up/down votes were not appropiate because  down votes are unlikelly to occur (thus the idea of rating the submited  work) and that we need to make it simple for peers to evaluate each  other work (thus the idea of providing rubrics with the badge  challenges).
  • In terms of priorities this kind of assesment logic goes just after the stealth assesment
• Guru assesment:
  • Some badges will require us to be more restrictive with repect to who can evaluate a canditate (i.e. trigger a badge award).
  • This  restriction will take place either by requiring that people evaluating  to already have those badges or that they have some role (course  organizers, badge admins, maybe mentors).
  • In other matters this kind of assesment will work like the peer assessment.
  • in  the case that we require users to have the badge to rate others the  initial seeds will be provided through honorary assessment (see below).
• Honorary assesment:
  • For  seeding purposes, we will need a way to issue a badge to someone  directly even if they have not completed a challenge (see guru  assessment for the reason we need this)
  • Badge  admins will be able to do this and the evidense will include who  awarded them the badge and why (text filled out by the badge admin).

BADGES

• For  badges we need the information that will go into the OBI (name, short  description, image) and a little more (long description) to display in  the badge page which url will go to the OBI when a badge is awarded.

BADGE CHALLENGES

• Creating a badge challenge will include:
  • Filling out a title and description.
  • Selecting and configuring the assessment logic.
  • Chossing the badge that will be awarded to those completing the challenge.

WORK SUBMISSIONS

• Some badges will require candidates to submit a piece of work for review.
• Submissions will use rich text (meaning users can embed things like github gists, ... as part of their submition).
• we  need versioning (like we have on tasks) for submissions because users  can go back to improve their work (don't think we need to reset the  votes after the user posts a new version if it is clear which votes  correspond to each version).
• users  should be able to "delete" a work submition (and restore it). data will  remain on the database but it will not be displayed. -- the # of  deleted submissions and versions can be included as a metric too
• if an assesor votes again in a different version the old vote will be voided in the count but still appear on the evidence

AWARDED BADGES

• It's was goes into the OBI.
  • data from the badge (name, short description)
  • the badge challenge (url that points to the page with information about the criteria for awarding the badge)
  • evidence  (url to a page that has the evidence of how the badge was awarded.  depending on the badge challenge and the assessment logic this will vary  but as and example it could have all the submissions from the user and  the ratings + feedback (s)he got)

WALL ACTIVITIES

• Activity  around badges will be on the existing 'Default', 'Learning', and 'All'  filters of the walls (groups/courses, dashboard, profiles)
• A new filter will be included just for activity related to Badges (named 'Badges').

BADGE CHALLENGE ENDORSMENT

• As  we said before challenges will have a default  endorsment depending on  who created them (someone appointed by a school,  p2pu or inside a  course).
• If  we implement support for badge challenges that created inside study  group courses, as next step we should provide to badge admins:
  • a way to endorse those challenges so they are marked as endorsed by the school)
  • a way to group/course badge challenges to the list of school challenges.

METRICS

• We will have to report metrics to evaluate the use of badges in p2pu (to superusers and school organizers)
• It can include:
  • # badges issued.
  • for  those badge challenges that require candidates to submit a piece of  work, # of submissions and number of people that submitted work
  •  for those badge challenges that requiere people to rate submissions, # evaluations and average rating