Badge and Assessment Design and Advisory Group
++++++++++++++++++++++++++++++++++++++++++++++++++

Call in info:
US Toll Free: +1 877 395 2347 **Please only use the toll-free number if you really need to*
US Local / International +1 415 763 5901

"NEW" Doc (V2, this has been shared since Nov 2011)
https://docs.google.com/document/d/1iYbHTj35SbtOgolTf3k1z7tiYEHZuzv7w4RVOdw9LEc/edit

Philipp/Erin v2.1:
https://docs.google.com/a/p2pu.org/document/d/1EJe36jE1sXsNpl_Yv5mwnVEi0DP87tP0rx9SmHd0sbQ/edit?authkey=CPbOmLQJ&hl=en_US

OLDDoc ("The Original"): 
https://docs.google.com/document/d/1TCUt9wD6OA-eIvtsXf6NIawI_MH11SWB83qDdZqQQZw/edit?hl=en_GB



++++++++++++++++++++++++
Sharing of the article beginning Nov 4
By Nils

++++++++++++++++++++++++++++++++++++++++++++++++++

++++++++++++++++++++++++++++++++++++++++++++++++++
Weekly Mtg
Jan 11, 2012

Attending

Discuss proposed next steps:

AECT [more narrow focus / instructional design]
EDUCAUSE Quarterly [widely available and read but would need to be shorter]
CACM
"First Monday" [no peer review / feedback]
Euro Journal of Open and Distance Learning
There is an IFIP meeting in England coming up - deadline for submission is soon - that is more international...and so a conference presentation and then following journal JCAL article is a possibility.
Several Jounrals liike CITE, JTATE, etc have a narrow focus on teacher education and technology

Discussion / feedback / thoughts on latest changes


Weekly Mtg
Dec 21

Attending
Nils who is looking for Alex who appears to be in the document
David sent his regrets

Philipp suggested (by email) that we congratulate ourselves and he will look for a means to get the document an editor and a single voice. He suggested we meet Jan 11, 2012 to review the work of the editor.





Weekly Mtg
Dec 13 (a tuesday because the 14th won't work)

Attending (via a google hangout)
Nils
David
Alex

We worked all the way down the document, resolving most of the comments and leaving a couple with questions.

We plan to meet the regular time next week, Wed 21, 10AM Pacific, but need specific guidance.

A remaining question is to Philipp regarding getting support for an editorial 'once over' to bring all the writing into a single voice. 


Weekly Mtg
Dec 7

Attending
Nils
Alex
Chloe
David

Alex did some wordsmithing and work in the front of the document

We decided that we need to very explicity say that when we use "peer" we don't mean other students as in a clasroom model. We simply mean a non-hierarchical relationship among learners -- anyone in the CoP at any level of expertise. 
Let's reiterate "peer" in different parts, emphasize what we mean by referring to the definition. David and I felt that at least one reader who kept returning to classroom model and typical school meanings of peer. (See notes from Nov 30)

User stories:
Should we use real people instead? That would give us more credibility, however we are not suggesting that they are representative of an ethnographic research.
Should we drop the user stories? They add to the document.
Authenticity adds to the document.

TASKS

QUESTION: Philipp, we have some concerns that the document does not in a consistent voice and wonder about giving it an editorial going over.


Weekly Mtg
Nov 30
(last week got cancelled because of Thanksgiving)

Attending
Nils
David


QUESTION: Do we need to very explicity say that when we use "peer" we don't mean other students as in a clasroom model. We simply mean a non-hierarchical relationship among learners -- anyone in the CoP at any level of expertise.

Observation: We used Google+ Hangout for the audio channel and it worked quite well when we stopped sharing video of ourselves. Video was nice to see David's friendly face, but after that he and I were really focused on the text of the document so it was less necessary.


WEEKLY MTG
Nov 16

Chloe
Nils
David
Carla
Alex

Discussing plans for getting more feedback

Current Feedback:
June Ahn: More concrete examples, how do we actually do this? Maybe reach out to people who can provide us with case studies?
Gary Brown: its not adding a whole lot of new things, maybe we are trying to get it in the hands of a new audience instead of breaking a new ground? For those new audiences we should be really clear about what we are adding.
Jayme Jacobson: clarification on the audience

Reflections:

Adding notes in the document, leave marks, so we invite commenting.
Question: are we writing it for a handful of 4-5 people? the Hewlett foundation? (philipp question) Should we cut it down 1/3? LESS IS MORE :)


Next Steps
Nils: call Gary and Theron regarding their comments to get clarity about their comments. Try Steve Ehrmann who was program officer at Annenberg/CPB & FIPSE, now retired. Review section 8 on evolving the assessment system. 
Everyone: ask others for feedback, reach out to doctoral students
Chloe: reach out to Philipp > scoping the audience

WEEKLY MTG
Nov 9 - cancelled

WEEKLY MTG
Nov 2

Alex
Carla
Chloe
David
Nils

Feedback on case study: Matt case study addressing all 8 components. Do we need all 3 case studies?
What do we gain fron the other examples; diversity of learners with different types of needs.
To show that it's not only about web dvelopment. The wider the audience the better people perceive it.

If we have three case studies we don't have to necessarily hit all eight points in all eight studies. Reads better without the notes. One thing to do before we get rid of them would be to match them when we create mentions in the different sections. 

Feedback on section 7 : include learner characteristics, review together

Who do we share it with?
open url
academics? global public response? both
first phase: share draft, community involved in the review of the document
develop this repository of ideas to 

How does the document get published? Digress.it commentpress, etc. Broad: Short version to Educause,More detailed in assessment journal...

Question for Philipp: what do we do next? after reviewing it where do we publish? is the paper only tied to Hewlett Grand? 

Next Steps

Nils: Review area around the Donald Schon reference and decide to expand the pp or delete it
Alex: will clean-up document, review conclusion
Chloe: edit 2 remaining case studies
David; will look at conclusion section
Email checkin in 24 hours, get some outside reviewers
Each of us think of couple of peers to review

WEEKLY MTG
Oct 26

Attendees:

David suggested working "backward" from Chloe's document to ours. She organizes the 8 points we made into a structure with concrete examples and guidelines. Our document would benefit from more examples, Chloe marked some places to consider. We may want to borrow from Chloe's document to the Framework doc.

Nils will tackle conclusion

Case for peers section needs work to be more effective

Contours of the Learning Community - needs elaboration to help Chloe -- Alex will try to trace out how assessment goes into this

Knowledge component + practice component (plumbers+craft) - unique vocabulary (speciailst language) + methodologies

Walk the whole document and tie back to deeper learning and assessment & tie back to the case studies.

Next Steps
Alex: make small tweaks, add more text on / section 7
Nils: Conclusion
David: Clean up Case for Peers / take design guidelines and link to main doc (examples?)
Chloe: link user stories to 8 components

WEEKLY MTG
Oct 19

Attendees:

Discussing V2 document
David, added new pp's in G & H
Section above there was too long. Chloe & David proposed moving to an appendix
Keep numbers A,B,C  
There are 3 heading 1's. - Philipp is re-arranging
David write a short version of the evidence centered design example

NILS: work on the 7 (8) items in the Exec Summary, round off the rough edges.

Examples: David to edit his current long example down; Chloe to point at places where she'd like more examples
Conclusion: Philipp says "connect back to work Chloe is doing, tying theretical framework back into design guidelines and implentation of the guidelines. We took this big step thinking about assessment in these environments, here are pointers and suggestions on how to move toward implementation" 
In conclusion bring all 8 principles together. These are a design guide guidelines. make it an invitation to next step. Wrap the concepts together again
CHLOE to share a design guide document she is creating. 3 overarching dimensions:  social,dynamic, personal  https://docs.google.com/document/d/1qWC4TzCGZI7i-NEeVAFtoAtXB3fSQeJKCi5rSVDSURw/edit
DAVID to share Seminal document by John Seely Brown early 90's. They outline online social spaces that support communities of practice. Bridge to our title, need assessment as an element of the spacce -- how assessment drives it forward. Commnity and Assessment are different; both needed to support deeper learning (both helping the learning happen and documenting that it has happened)

P's thaoughs on relationship between design guide document (Chloe) and assessment framework document (trio). The two documents should test one another. 

Next steps:
Table of Contents: philipp (and others by chaning the headings and sub-headings)
Chloe to add pointers were examples will be helping
David to narrow down example for SoW
David create appendix
Nils: Draft the Conclusion

WEEKLY MEETING
Oct 12
Meeting Cancelled
Nils forked the document (see new URL above) and did clean up work as proposed


WEEKLY MEETING
Oct 5
Meeting cancelled because several people were absent


WEEKLY MEETING
September 28 1PM ET

Attendees:

Agenda:

Alex's Proposal for Forward Motion:

1. Some agreement on 7 +/- principles (Alex's 7 paragraphs). Do we have that? Yes, I think so. Alex & I just reviewed (with wordsmithing, potentially)

2. Doc Structure:

a) Short intro 

b) Principles  a & b = executive summary that we have now
b2)    a short concrete example here (user story) - CHLOE

c) Deeper learning yes, and serves an intro to why peers - NILS

d) P2P -> P2PU build the case for peers as path to deeper learning, P2PU as an example implementation -- NILS

e) Discussion / Lit Review for each Principle

f) Proposed framework for assessment - this is where the pieces come together. David has some of this. Show how implementation notes come together in a whole system -- DAVID
    Audience for framework for assessment = designers,educators, online community?

g) Conclusion (including assessment of framework?) relative ease of understanding how this works in informal learning vs the barriers of formal education structures

2. Chloe will work on pulling together an exemplary user story (or 2 or 3)

3. Nils will pull together C& D

4. Alex will thresh out, reorganize section E as 7 subsections. Some redundancy, but can be worked out!

5. David will get F organized to draw together the 7 into a coherent system, and compared/contrasted with other frameworks of assessment.

NEXT STEPS
CHLOE: will work on pulling together an explary user story (or 2 or 3)
NILS: will pull together c) & d)
ALEX: flesh out, reorganize section E as 7 subsections. Some redundancy, but can be worked out!
DAVID: will get F organized to draw together the 7 into a coherent system, and compared/contrasted with other frameworks of assessment.


WEEKLY MEETING
September 21 1PM ET

Attendees:

Notes:

NEXT STEPS

WEEKLY MEETING
September 14 1PM ET

Meeting cancelled by mutual consent due to multiple schedule conflicts

WEEKLY MEETING
September 7 1PM ET

Attendees:

Notes:

NEXT STEPS:





WEEKLY MEETING
August 31 1PM ET

Report on outcome of David checking with Phillipp about Sharing of the Framework Draft. When?  -- Philipp says not yet, need more readable

Attendees:

Agenda


Notes:




WEEKLY MEETING
August 24 1PM ET

Sharing of the Framework Draft. When? Check with Phillipp. David will check. 


Attendees:
Erin
Chloe
Alex
Nils
David
Phillipp

Nils: Daniel Pink video around motivation - relationships to P2PU and this work: http://www.fastcompany.com/1646337/science-shows-that-bigger-bonuses-create-worse-performance

automony (nature of the learning activities design), purpose and mastery as incentives of performance (Pink - video) - connects to P2PU - that instructional staff is voluntary...and for learners, how the badges system can embody these sources of motivation.

how to utilize strengths, interests and aspirations (all as forms of badges, or as supported d within the eco-system of badges - to create Purpose)

Forms of Learning to mix and match with strengths, interests, aspirations:
Exploring, Tinkering...as badge paths / mechanics for learning
"Explore" 
"Tinker" Trial and error - building and testing
"Building/making"
"Problem solving?",
Hacking
Hanging out, messing around, geeking out (ala Mimi Ito)
"Assess" (e.g. literature review)

Alex: blog post on boy/eagle scout badges
It was more the process/experience - getting connected with mentors, etc.
Examples we see now lose that process - how can we bake in more of that process?
intrinsic v extrinsic motivation is not as clear of a delineation - the two can go hand in hand
motivate people not only to learn the things that they are there to learn, but also engage in the assessment as well (since we are mainly working with assessments)
Re achievement systems: something to check out is a collection of short papers in the game studies issue of last Feb, it's all about achievement systems http://gamestudies.org/1101
>this one in particular might be helpful, Re:process http://gamestudies.org/1101/articles/medler
What are you good at, what do you like to do, what do you want to do in your life - looking at intersection
badge system should reflect those interests, self-driven learning pathways/plans, etc
helping people to locate their passions (aspirational badges- what do you want to be good at, what is the community you want to join)

Orientation:
Anya K - DIY learning plan - launching on August 30 with small group
For SoW up in the air still since the model is changing slightly - might be that the orientation/learning plan should be a challenge
Nils is volunteering to get into that loop - Erin will follow-up with him about how to get involved in those conversations

Chloe:
Starts Septemeber 1st
Helpful for her: use cases for learners and organizers / experience / taxonomy of forms of learning (i.e. tinkering, exploring)

David: 4 steps of a cycle - (can jump back and forth among the 4 quadrants)
plan - gathering direction and forming purpose, aspirations , decide time scale 
do/act - implement the plan. tools to help action
study - review & reflect. check on your plan. check on external context. how far have you come. is the plan aligned with the context
decide -  on the next steps in the plan.

Paper:
A lot there - step back and see where we are at, where we need to dive more deeply, where we need to back off
Alex: fleshing out his section over the next couple of days, then will see where it fits with the rest of the paper

Goals for next week:
Do some building out/wrapping up of sections, then read through the rest of the paper
Next week will discuss how to start stitching it together - where there are gaps, inconsistencies in granularity, etc.
compare our thoughts around the paper in general - where we think it should go
Chloe read through the document as is and report back on usefulness


++++++++++++++++++++++++++++++++++++++++++++++++++
WEEKLY MEETING
August 10 1PM ET

NOTE: Next week's meeting is cancelled - too many conflicts

SoW Charter http://commonspace.wordpress.com/2011/01/18/draftwebcraftcharter/ Erin to send another if this draft is out dated.

Who is doing the assessment (peer, expert)? Why are they motivated to do the assessment?
Does assessent (by peers) serve as a mechanism to build community quickly? How does assesement help with driving participation?

Assessing participation in P2PU to encourage participation. Different than the learning validated assessment.

Expert coming in pays P2PU in effort (contributing quality assessments) in leiu of cash and P2PU is source of credential for that person. (This thought came from talking about Western Govenor's U where the student pays cash to take the assessment and earn the credential.)

Attendees:
Erin
Chloe
Alex
Nils
David

Updates/Progress/Notes:

Who is creating the assessment?
How to incent the people to perform the assessments


Next Steps:



++++++++++++++++++++++++++++++++++++++++++++++++++
WEEKLY MEETING
August 3 1PM ET

Attendess:

Nils will be absent, but notes that he made progess on the Metrics of Framework Success section in the Google Doc.

Contract:

Progress to date / Notes:

Next Steps/This Week:


++++++++++++++++++++++++++++++++++++++++++++++++++
WEEKLY MEETING
July 27 1PM ET

Present:


NOTES:
Framework Google Doc:

Design ideas (SoW) - where should these go

For Chloe (implementation):


++++++++++++++++++++++++++++++++++++++++++++++++++
SOW DESIGN PLANNING MEETING
July 25 1PM ET / 10 AM PT

Agenda:

NOTES:
Thoughts on voting/rubrics:

Assessment Design:

Task model:


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

KICK OFF MEETING
July 19th, 2011

Attendees:
Erin, Philipp, Chloe, Nils, David, Alex

Hewlett Grant: looking at deeper learning - how to assess and provide evidence for

Deliverables:

Hewlett's Definition of DEEPER LEARNING:
a combination of the fundamental knowledge and practical basic skills students will need to succeed in a fiercely competitive global economy. Specifically, our definition of deeper learning brings together five key elements that work in concert:

Draw in the 21st century learning - be clear that we are redefining these for this environment

SoW Design Phase

Framework Mapping:



where do badges fit in? might be its own system
need to have a name for ours

2 groups granted millions from the government to rethink assessments (for K12):
mimi's connected learning group is connected
can we hook into these guys? at least good to mention in the document 
1) SMARTER balanced assessment
2) Partnership for Assessment of Readiness for College and Careers, or PARCC
TASK: everyone reach out to their network to get more info on these guys

PROCESS

POST-it Notes from Badges Mtg II (July 18-19)
Post-it notes were grouped by the participants into groupings, two of those groups are reproduced here as they may shed light on design questions or framework ideas.
Nils Peterson editoral comments while posting these notes in [ ]

BADGES FOR LEARNING

ASSESSING BADGE SYSTEMS
This heading was also the topic of a breakout session. The original post-its were augmented with new ones and organized into a structure

Guiding Questions:

Responses were classified into 4 groups. Group #1 was giving higher weighting

Group #1

Group #2

Group #3 

Group #4