Purpose
Starting a conversation between University of Michigan and P2PU to conduct research through the DML competition around the P2PU badge system currently under development. U-Michigan is interested in learning more about how badges are created outside traditional competency frameworks; also interested in overall learning trajectories and self-directed learning processes.
Background/Resources
DML research competition: http://dmlcompetition.net/Competition/4/research-development.php
P2PU DML project application: http://dmlcompetition.net/Competition/4/badges-projects.php?id=3201?
P2PU research questions of interest (scroll to "burning research questions": http://pad.p2pu.org/p/community-20120719
Open.Michigan badges wiki: https://open.umich.edu/wiki/Badges
Competition
"DML seeks research proposals that support and inform the design, development, and deployment of the [P2PU badge system, a winner of the Badges for Lifelong Learning Platform award]."
choose a type of research:
- Collaborative design research with [P2PU], where researchers would be involved in the prototyping, testing, and iteration of badge or badge platform development. Proposals of this nature would need to include a letter of support from [P2PU]. (seems very system/development/process focused)
- Research with using the design, development, and deployment process of [P2PU], to examine the general effectiveness of badges and badging systems to motivate, recognize, and assess learning. This can include analysis of designs, or conducting interview or survey work with competition winners. Proposals of this nature would need to include a letter of support from [P2PU]. (this is much more about the learning)
- Research that studies or synthesizes research on existing or historical badging, reputation, assessment, or credentialing system in ways that can inform the design, development, and deployment of the Digital Media and Learning Competition winners. This can include developing theoretical insights, design principles, or conceptual frameworks. (a broader area that need not focus on P2PU or current DML competition winners)
Potential Resources:
Piet works with a social scientist for some help with potential interviews and qualitative data analysis (Airong Luo)
Interviews with P2PU staff
research questions posed by DML:
- How have ranking, badging, reputations and achievement systems been used in games, clubs, competitions, and other forms of interest-driven activities? What design principles and guidelines might we glean from past and existing cases that can inform the development of badges for learning?
- What role have accreditation and certificates played inside and outside of formal degree programs, including areas such as core curriculum, work skills training, arts, crafts, and other trades? More specifically, how might badging help to address some of the challenges currently facing teacher assessment and credentialing?
- How have learning institutions, groups, and individuals produced, utilized, and exploited various credentialing and reputation systems? How has such credentialing been changing with the shifts to a digital and networked society?
- How do badge creators define mastery? To what degrees are the competencies represented by the badging system and individual badges clear to the learners? (interesting for P2PU?)
- In what ways is mastery assessed? Are learners given productive opportunities to demonstrate mastery (in their application, in producing and not solely consuming knowledge, and in their participating in learning and knowledge production)?
- I'm pretty interested in this question--but bristle at "mastery"
- Requires working with RAND project--which would make it a very different project
- How do badging systems conceptualize and operationalize learning pathways/trajectories? Do badging systems offer opportunities for learning connections and interactions with others, as well as for feedback? For leveling up along the learning trajectory? How or to what extent are novice to expert trajectories made available? (really interesting for Piet, Michigan)
- How is the badging system conceiving and operationalizing validation or legitimacy of the learning taking place, and so too of the badges being issued?
Next Steps
Eligibility? (VMG is checking with Sunny)
Decide on a research question (but we have some leanings)
Piet to write up how he thinks this research could benefit U-M (Coursera group, Med School)
- As certification of learning becomes more distributed and less focused on academic centers, major academic institutions are doing some soul-searching to understand and articulate the value they bring to society and individuals in the short and long term. The University of Michigan has experimented with a variety of programs and initiatives aimed at building and defining a unique "undergraduate experience" for its students, many of which are extra- or co-curricular. Additionally, the professional schools and colleges at U-M are exploring co-curricular tracks for interest-based specialties and already have a history of certificate-based programs served both in-person and online. Recently, U-M has entered into a partnership with Coursera to provide high-quality, asynchronous course-based learning experiences to learners all over the world, for free, but without robust certification.
- Understanding how badging, reputation, and achievement systems work in these interest-based realms can help the University define its portfolio of experiences and more appropriately brand and market those according to individual needs and uses. The motivation and learning components of these systems may be the key to differentiating experiences in academic institutions. Knowing how and when to apply systems like badging or higher level credentialing (degrees) would then be crucial to diversifying the University's programs and offerings.
- But I care about the learning.
- Learning is much more social and connected than it used to be, or maybe it's just coming out of the trough of industrial disillusionment. Regardless, we have an opportunity to recognize the various ways learning takes place and help people make sense of that. When people understand themselves and can articulate who they are and where they want to go next, they have an easier time integrating themselves into their environment and engaging with the world.
- What I want to understand is the complexity of an individual's motivation vector around specific tasks and processes, how that vector is tied to individual identity, and how what they do creates a flurry of feedback that is then integrated back into one's identity. This is my description of learning. And I think it's more important for people to be able to describe who they are and show their growth than it is to show individual achievements and aspire to existing roles in society. Badge systems can help show growth and trajectory. They can be used to build contextually aware individuals with complex, yet malleable identities.
Luka's Notes--on Thursday after the call
Letter of Intent (if we want to) by August 27h--that's priority, but rolling acceptance & based on grantholder
Piet is back to work on August 20th
Eligability--from Sunny Lee, August 7 2012
I am a former Digital Media and Learning Competition winner. Can I apply?
No, you cannot apply as a primary applicant. However, you may contribute as a team member to a research project seeking Competition funding.
I am already receiving funds from the Gates Foundation. Can I apply?
Yes, but only if you are not or have not been the primary applicant for a Digital Media and Learning Competition project.
I am already receiving funds from another source. Can I apply?
Yes, however, any award you receive from this Competition should be applied to the part of the project that is not being funded by other sources.
Email from Luka (8/21):
We had talked about looking into this trajectory question, the narrative people have of learning, how that can cut across class/ability/culture in general.
We know from work in traditional education that what is called your "habitus" (a "structuring structure") impacts how and what you learn. Bourdieu (http://en.wikipedia.org/wiki/Pierre_Bourdieu) developed the term habitus, which is basically a taken for granted set of dispositions, taste patterns, etc. Your habitus is based on how much cultural capital you have (economic capital = money, assets; social capital = your networks; cultural capital = your education, your learned dispositions, knowing to pick up the right fork or say the right thing, etc). In traditional education settings, higher cultural capital students do better. I can break this down for you more later, but kind of trust me here if its not making sense. I've been really interested in understanding 1. what is cultural capital in open education 2. how is it structured and how does it structure the space 3. are there ways that AI or good design can help people learn the right cultural capital so they can succeed in the space.
- another thought I've had around this area is: does one's cultural capital determine, early on, one's access and use of open education opportunities? in other words, is "open" only available to those in the know? (social mobility vs. social reproduction)
I think this is where we have a project. There's already several platforms out there that use AI to determine taste patterns, like www.hunch.com. (see their "about" section: http://hunch.com/info/about/)
I think we can build a really awesome team of social scientists, open ed gurus like yourself, and developers in the hopes of:
1. designing a methodology that can discover what cultural capital is in open education (I'm already on the way to developing this with my research)
2. designing an algorithm that can predict choices and trajectories in the open ed space (we'll have to use p2pu here, but can brainstorm how to reach across platforms-my activity diary research may be a lead) agent based modeling - how do we want to do predictions? ABM allows for interactions and complexity than traditional models
3. develop a platform/dashboard/interface (here's where I need a good developer to flesh this out) that can use that prediction tool to help learners create a narrative of their learning trajectory and help them a. gain cultural capital relevant to the space b. make better informed decisions about their trajectory and
4. (this is where badging comes in) help inform existing badging systems that currently reward on a post hoc structure (you complete, you get badge)
- recognition badging, not reward badging - way more important and conducive to identity work
the funding agent that is interested is the Institute for Policy Studies.
www.ips-dc.org
They (and I) do a lot of "new economy" work (aka how do we prepare our economy for climate change, societal change, etc) and they are interested in bridging into open education. My advisor is really close to them and I've done work for them in the past and have a small reputation in the new economy field. We would have to make the case that this project fits their work, but I think as long as we stick to the idea that our research and development has the aim of making open education "more open" we're all set.
next up, putting into 250 words
Roles:
- Piet - big picture thinking, strategy from the beginning, 1&2, also really familiar with badging, has done some interface design, will be helpful with feedback for designer; has done some agent based modeling work as well, happy to work with Jon; also good with writing (p2pu - relating back to them)
- Jon - 1 &2, more 2 - ABM, prediction side
- Emilie - step 1, cultural capital - theory
- Luka - big picture thinking, steps 1 &2 , writing, working with funding agent (relating back to new economy movement)
- Evann - algorithm
- Sabrina? - cultural capital
Luka - take first pass at letter of intent
- send Piet ed studies on HCC students doing better
- send blog to Piet, plus any other writing
Piet
- - dissect letter of intent and add a bunch of jargon that makes no sense (so we can dissect it again)
- - send Luka ABM course, background
Letter of Intent - 8/27/12
A critical question asked of open education is whether or not it is open to everyone. Sociologists of education have long cited ways that education systems can reproduce existing social inequality. Open education systems have the potential to disrupt the reproduction of social inequality in education through free and open content and peer learning. However, the same mechanisms of cultural, social and economic capital that reproduce inequality in traditional education are at play in open education. Luka Carfagna, a sociologist from Boston College, has conducted preliminary qualitative research on open education platform users to develop an understanding of how cultural capital can influence learner trajectory. Piet Kleymeer, an open education specialist at the University of Michigan, has studied badging systems and their impact on learner trajectory in open education spaces. Together with the support of a third-party organization, we plan to help P2PU platform designers understand how badging can be used to reconceptualize learner trajectories that are sensitive to variant levels of user capital. We propose a three-stage project that begins with research on how capital, specifically cultural capital, is showing up in P2PU user experiences. Next, we will build an algorithm that predicts capital input’s effect on badging outcomes and learner growth. Finally, we will team with developers to design a dashboard that uses this algorithm to better advise learners on how to construct a learning narrative using known successful trajectories on the platform, with badges as a central metric of trajectory.
Finding a title:
- trajectory
- cultural capital
- predict
- badge
"Opening" Open Education: Understanding the effect of Capital on Open Learning Trajectories+1
Cultural capital and the open learning trajectory
Understanding the effect of capital on learning trajectory: Badging as equalizer
predicting capital input’s effect on badging outcomes and learner growth
Equalizing open education through badging
Predicting learner growth through cultural capital
Destructuring of open learning...robots with missles in their eyes.
"Opening" Open Education: Sensitizing badging to capital's effect on learning trajectories
Comment from acceptance of LOI:
To be eligible for judging please address the following in your Final Application: Your proposal walks a fine line between a development grant and design based research, as such your Final Application should focus on understanding how cultural capital can influence learner trajectory and how badging can be used to reconceptualize learner trajectories that are sensitive to variant levels of user capital.
- Luka - track down someone to parse this out (also ask about letter, how does this process work)
- model rather than predict - re: algorithm
- do we need a letter of support (probably - Piet will email Vanessa and Philipp)
- how can cultural capital influence learner trajectory?
- how can we use badges that are sensitive to variance in capital to reconceptualize learner trajectories?
Questions:
- should we be a dev grant or a design-based research grant? (latter, I presume)
- is this saying our application needs to outline our current understanding of how cultural capital affects learner trajectories and focus our future work on reconceptualizing those trajectories through badging?
- my stomach hurts... why? (unrelated, but important)
- final applications due October 1st, 8:00PM ET
- Proposal outline, including rationale and work plan, no more than 1500 words (Luka will take crack at this, Piet will send notes to Luka)
- Preliminary research: built on LB's work with understanding cultural capital
- 13 interviews done
- 15 hrs per week of observation (on-going) since Nov 2011
- no direct measures of cultural capital (no surveys; can be intense)
- more focused on defining a loose cultural capital spectrum
- badging- used in so many contexts, so many ways- diluted quickly
- we will need to describe what kinds of badges we care about and why
- assessment is scaffolding perception process
- but assessment based on more strict role standards
- Public forum outline, no more than 250 words
- Budget and budget narrative for proposal
- stipends per person
- transcription
- travel
- interviewers
- IRB (with IPS?)
- software
- Budget and budget narrative for public forum
- Letter of support from your prospective DML Competition grantee research partner (if applicable)
- other logistics
- Piet needs to run this scenario through his grants office to make sure we don't screw anything up
- must attend: Digital Media and Learning Competition workshop to be held for awardees at UC Irvine on January 24-25, 2013;
- must attend: The Digital Media and Learning Conference in Chicago, March 14-16, 2013. Registration and travel costs for both are to be included in the awarded budgets.
- - IPS (Chuck Collins), wants to do this but needs to be pitched to DC folks
- how will the disbursements go? one-time check
We can break the diagram into two adjacent spaces: the self and the environment. The idea is that the self (or selves) has a loop that connects its actions with the environment and then senses and interprets any outputs from the environment. The self then analyzes what it perceives based on its understanding of standard roles it plays in society. Today's roles are malleable and evolve over time as different cultural standards interact with each other. These roles have been adjusting dramatically as new cultural capital comes into play. So the self has a perception of its current role based on the feedback it interpreted from the environment and then it has a projected identity made up of both cultural norms and a motivational vector. Typically (always) there exists a difference between what is perceived and what is projected. This difference becomes translated into an intended action by the self. That activity then interacts with the environment and the ensuing situation results in more output to be sensed, interpreted, and analyzed. The cycle continues.
Some interesting notes:
1) our perception of the feedback we sense is often inaccurate (citations to self assessment, others?)
2) the standard roles we used to subscribe to are disappearing or morphing quickly
3) the activities we engage in in a learning environment are often structured to produce specific outcomes based on competencies within standard roles; similarly, those outcomes are assessed as a proxy for preception, essentially scaffolding the perceptual control process
4) if we have error between our perception of our identity and reality (delta-p), and error between role standards and our identity projections (delta-i), we end up with some error in our intended actions (delta-a); over time, we strive to reduce delta-p, hence the name "perception control".
5) typical education systems have created a perception by-pass that forces individuals to focus on assessments of outcomes rather than our natural comparison processing of identity perception and projection. meanwhile, the education systems have not updated quickly enough to match changes in both role standards and how different roles are becoming privileged over others. so the assessments and the standards introduce more error into the control system, resulting in intentions that are completely out of line with what the environment would suggest is productive.
6) badges that continue down the "assessment" path will do nothing to affect this cycle of increasing error; in fact, assessment badges may even hasten incorrect identity shifts
7) we suggest badging be connected to this control system in two ways: a] use badges to help individuals make sense of the changing and multiplying roles in society (reducing delta-i); b] use badges to provide timely and relevant feedback to individuals, improving their perception and reducing perception error (delta-p).
9/21/12
logistics:
- -DML wants us to focus on research only - no development side. They couldn't answer any further than that, since it is a "competition"
- -Do we need to name IPS as fiscal sponsor in application? If so, there's a bunch of paperwork we need to do. Let's talk about this.
- - I need the name of the IPS sponsor and contact info for my grants office - EMAIL SARAH's INFO TO PIET (Sarah Byrnes: Economic Justice Organizer at IPS); they want a 15% fee from the award
- - technically graduate students cannot be PIs/primary applicants in the DML competition; are we violating that? FAQ: "Can graduate students be primary investigators (PIs)? No, students cannot act as the PI on a project, but are welcome to be collaborators." -- this shouldn't be an issue
- - IPS needs us (P + L) to be an organization if we mark IPS as our fiscal sponsor on the proposal application
- - we need to run the grant through an org but unsure when this needs to be finalized
- Don't need to identify IPS as fiscal sponsor in application
- -need to grab letter from Phillip/Vanessa
- -do we want to solicit a letter from any other organizations? since p2pu only can work with one of us, should we pitch to someone else as well and increase our chances?
- look through list, if there's an awesome match then lets reach out
Proposal outline:
- trying to create a research design that answers two questions:
- 1. how can cultural capital influence learner trajectory?
- 2. how can we use badges that are sensitive to variance in capital to reconceptualize learner trajectories?
FIRST QUESTION - HOW CAN CULTURAL CAPITAL INFLUENCE LEARNER TRAJECTORY?
What we know (traditionally) about cultural capital and habitus
· A “structuring structure”
· Plenty of studies in education that show it stratifies the education space, and consequentially the employment space
·
Open education- is it open for everyone?
· We’ve framed this study within this question – will open content, artificial intelligence, low entry costs, etc = “opening” up of education
· Two things to consider
o 1. This cultural capital thing. We (Schor, Dubois, and Carfagna forthcoming) started talking about “digital habitus” in the timebanking paper. What is a digital habitus? Can we design the study to figure out what a digital habitus is and how it structures across class?
§ METHODS QUESTION
ú Usually this involves some SERIOUS deep study – in depth interview plus survey of class background.
ú Can we get creative here? Survey plus user testing possibly? Would only give us a “real time” perspective. In the past, Julie talked about tracking software. I’m trying activity diaries (haven’t yet designed)
- send alerts, request response- text message(automated), ask them to respond - bridges between tracking what they're doing and days into larger report
- but will assume level of literacy
- ** instead of having full methods plan, have idea of types of data we want and ideas of how we would get it**
- what types of data do we want? (ideally)
- class background, lifestyle
- attitudes towards knowledge/learning/education
- attitudes and behavior with technology
- idea of what learning trajectories look like for different groups of people
- ideas of what signals employers/others are looking for
- understanding where those shifts in signaling are happening
- and it which groups it matter
- better understanding of identity
- how cultural capital informs or constrains the identity/ies of people
- how badges are influencing learner trajectory currently (or missing out on understanding learning trajectory)
- how do we want to get it?
- contextual inquiry - brand of ethnographic research
o 2. Changing society/economy
§ Richard Florida says we have three classes (creative, service, working class) – I think he left out the finance guys or maybe what we’d call an executive/professional? class
§ Working class jobs are going away
§ Service class jobs are plenty but awful
§ Creative class jobs are stratified between high pay/low pay, stable/unstable, gig/full time, etc
§ Big call in popular readings of this stuff to teach entrepreneurship, curation, etc in order to a. pull more people into creative class and b. create more value and stability in creative class (EMAIL MIMI ITO re: creative cultural capital)
· Ultimately, creative class calls for a different habitus and different repertoire of privileged cultural capital
o There is evidence (ask mimi for this, re: discussion during macarthur webinar with skillshare founder) that this habitus and cultural capital still maps onto existing ideas of privilege and will stratify us the same way
o BUT there is more room (we think) for change – more people are finding ways to be entrepreneurial (is JBF an example? http://www.jbfsale.com/ Tell Piet about it)
· So let’s get back to open education
o If we intervene, we can guide the FIELD towards a more inclusive and open recognition of habitus and cultural capital at play
o Also, if we intervene, we can guide PEOPLE towards better recognized activation of “skills” and “dispositions”
· How do we intervene?
o BADGING!!!! -- Ok Piet – this is where you get to blow my mind
- - people aren't really earning badges yet; or if they are, there's no place to display them, no good way to create your own easily or hand them out to people yourself
- - the first look at badging is mainly replicating current educational assessment structures; people aren't thinking about the way badging has, in the past, been more an indicator of social status and identity presentation
- - http://www.olpglobalkids.org/content/six-ways-look-badging-systems-designed-learning (frame 6 replicates 1 and 3)
- - http://openbadges.org/en-US/about.html (doesn't address that motivational vector is super complex)
- - the intersection of cultural capital, motivation and identity makes badging interesting; we will be able to use badges to more accurately display/represent cultural capital at specific times, in specific contexts, to specific individuals; we need to understand how motivation and identity mix with shifting cultural standards, especially as they relate to "what people do" (aka employment, but I want to keep this bigger than employment)
To do:
- Find Piet on Mendeley - DONE
- print notes
- evernote? Find Piet -done
- Piet to look through other DML badge projects and identify anything interesting - DONE
- - Design for America: A Badge Community for Innovation (my friend Anya is on this team) - I see some good overlap with peer-learning and the new capital being generated and shared by young entrepreneurs
- email Philipp about the letter (do we need to do anything?) - DONE
- - we don't need to do a thing; he'll send us the letter next week when he's done and we include it in our application.
- outline to Piet- walk through it tomorrow morning
- connect Piet to Sarah Byrnes
- Piet to talk more with U-M grants office - DONE
- - grants person is asking if we'd find it easier if U-M just held the grant... something to think about this weekend?
- + does this change the primary applicant? needs to be PI on the research.
- + would Luka be an independent sub-contractor through U-M? cannot get BC involved
9/22/12
1500 words is roughly 5 pages (double space)
· need to accomplish the following:
o research question/executive summary - 1.5 pages
o background and significance- 1.5 pages
o methodology – 1page
o work plan – ½ page
o bibliography (separate)
Research Question/Executive Summary – 1.5 pages
· Begin with 1 paragraph introducing question “is open education open?”
o Promises of open ed/learning
o Variance of projects/platforms/resources out there
o Why its happening, etc
o Leave reader with sense of doubt at end of first paragraph
· Why would we doubt openness? – 1 paragraph
o People usually answer this question with things like “access to internet” or “language barriers” or “time”, which are obvious issues that need to be addressed
o However, an under studied area is Cultural capital.
§ What we know about cultural capital in traditional education
§ preliminary research (Carfagna and Schor) – shows that cultural capital is stratifying the space (give brief examples)
o 1st research question:
§ How can cultural capital influence learner trajectory?
· Learner trajectory (PIET) – 1 paragraph
o In 1 paragraph, take us on a trip through why/how badging has been used to conceptualize learner trajectory
o End with our second research question:
§ 2. how can we use badges that are sensitive to variance in capital to reconceptualize learner trajectories?
Background and Significance - 1.5 pages
· The new economy – 1 paragraph
o How is the economy changing?
o Why do we need a systematic response from education?
o Importance of not reproducing existing social inequalities
· Piet’s feedback model – 1 paragraph
o Box diagram – can you break down into a paragraph that shows how we need to change perception and environment? Do I have that right? Reference how roles are changing re: new economy (creative class stuff here)
· Through social scientific research we have the power to better understand how this feedback loop works and interrupt it with technological innovation and educational pedagogy re: awesome badging – 1 paragraph
o In order to get there though, we must first ask these two research questions (list our two questions again)
Methodology – 1 page
· Data we’d like to have access to, possible ways to get to it (we talked about this yesterday, I’ll flesh out in outline/memo form shortly)
· Here we should mention who we will work with (other researchers, interviewers, etc) and what we might use (Software, etc)
Work Plan – ½ page
· Do we know what our timeline is?
- 1 year - starts january 1, 2013
- march - DML meeting (preliminary findings)
BUDGET
- half-time for Luka: $30K
- 40% time for Piet: $30K
- SXSW edu
- public forum; practical workshop through IPS
write up in etherpad or google doc :: https://docs.google.com/document/d/1Fn2gjmKZC2IOF31AMiFncoL9_aKzwA5tL42hrlXaSPQ/edit
deadlines -
9/24/12
need to make sure that proposal gives some reference to how P2PU will benefit from research - will need to make mention of this in first section and then go a bit more in depth in second section (will say more once those sections are written, but this is more just a note for myself)
http://info.p2pu.org/2012/09/24/update-from-digital-media-and-learning-duke/
