Challenge: Mashup for Data Visualization

VMG learning goals:
-Create an atlas of open historical, environmental and geographic data layered over a map
-Guiding philosophy: history is a series of choices, geography is always in flux
-Learn how to visualize data & create a mashup

Sources of data: http://www.gapminder.org/data/

JBs recommended steps:
1. Get a map on a page: using google maps, in html
2. Format your csv into json data
3. Write a js function that will add the data to the map

10/10/2011
Introduction and Lumped Abstraction 
video by Anant Agarwal - http://academicearth.org/lectures/introduction-and-lumped-abstraction
What is engineering? It’s the purposeful use of science
If we look at nature, there are large tables of data through the elements
Engineers-->apply layers of abstraction
A layer of abstraction is an action that succinctly rationalizes data or behavior
-an equation
-simple statement
-interpretation of what the data tells us

Take complicated things, build layers of abstraction, make them simple to build better systems
Systems evolve to have more and more complicated data inside them, but easier to describe
Example: building layers that take away from nitty gritty details to easier to use tools

(Extensive example) Data-->Maxwell’s Laws-->Lumped circuit abstraction--->Digital abstraction-->Combinational Logic-->Clock some aspect of time-->ISA-->Language abstractions--->Software systems language (ie Linux: a set of system calls)                                

The distance between nature and complex digital systems is a series of layers

10/12/2011
Up and Down the Theory of Abstraction: A Systematic Approach to Data Visualization
Bret Victor: http://worrydream.com/LadderOfAbstraction/
High level effects from a series of low-level interactions
Keep a balanced viewpoint of systems: both on the ground and bird's eye view
Start with the simplest design possible
Time: must be able to see both granular moments and the macro level
Algorithm--we can arbitrarily set parameters, like turning angle-->refine over time with tests
Looking across a system from all time, behaviors emerge
Abstracted over time-->noticed a behavior
Bring it back to a particular time-->back to concrete
"Every point on a visual abstraction typically corresponds to a particular concrete state."
Next level of abstraction-->both time and turning rate-->from there, abstract all algorithms
What does all of this abstracting do, for Christ's sake? It bringz out high-level patterns

VMG in search of a map

Data requirements
Population data
Historical political boundaries
Precipitation levels/environmental categorization (tundra, taiga, but more helpful so folks get what these mean)

Really really would like to find
Political history--leader/political party
Languages predominantly spoken: pull from Wikipedia? That's complicated with boundaries always changing
Material culture & artifacts--hunt for material culture in CIA World Factbook data (imports and exports are too vague)

Lesson 3: http://jsfiddle.net/mozzadrella/XErAz/21/

Q&A:

Lesson 4: Markers
VMG creates her own map in google maps "My Places"we'll do more with this later
Finds datasets of gepolitical coordinates (for later)
Tutorial... update your jsfiddle according to this
http://code.google.com/apis/maps/documentation/javascript/tutorial.html

Basically we're including the GMaps JS library, and creating a map in our page. Before you included an iframe which is just like showing a page within a page. For security reasons we can't easily manipulate it.

Once we have this new map on our page, then we can place our first marker. Place it in boston.
Places points on map...
Place markers using Javascript in JSFiddle: http://code.google.com/apis/maps/documentation/javascript/overlays.html#Markers

VMG Starts Lesson 4:
Getting tricked up on defining sensor bit as true or false--wouldn't it be false for this exercise?
But can't figure out where to put it.

A-ha! I GET it! You have to define it in your script. Documentation leads you to believe that you need to define sensor somewhere else, additionally. 

This documentation is written for robots.

OK, found my lat & long and tested it
VMG is stuck, keeps getting maroon tags in the jsfiddle that are not explanatory
When I try to close my tags, they become red and angry

I've researched other examples--don't know what I'm doing wrong:
http://jsfiddle.net/mozzadrella/CBfNN/3/

VMG Qs for John: 
1.) "HTML nodes are children of the JavaScript document object, and we obtain a reference to this element via the document.getElementById()method."
I thought the document object was the whole enchilada? How is the Javasript document object different from the whole document? Nodes? Children? 

Lessons from today's talk with John:
Reduce, simplify
Commas and lowercase count!
The documentation can be wrong
Know every line and what it means

Lesson 4 Markers: Round 2 (10/22/2011)
-"HTML nodes are children of the JavaScript document object, and we obtain a reference to this element via the document.getElementById() method."--->aha, they are indeed related. Refers to the information called up in the empty div
-This works, but not the way that John told me to: http://jsfiddle.net/mozzadrella/CBfNN/29/
-VMG will copy and fiddle with 

John Qs:
What's the difference between an object constructed and an object literal?
http://code.google.com/apis/maps/documentation/javascript/tutorial.html#mapoptions

Things VMG watched on her Saturday night with geocoding:
"Moving Beyond Markers: Advanced API Customization" http://www.youtube.com/watch?v=dV1GSIWimag

"Performance Tips for Geo API Mashups": http://www.youtube.com/watch?v=zI8at1EmJjA

Lesson 4 Recap:
Happymap: http://jsfiddle.net/mozzadrella/CBfNN/31/
Sadmap: http://jsfiddle.net/mozzadrella/CBfNN/32/

I found doing the task twice useful.  I could tinker with one "steady" jsfiddle without breaking it.  Which kept me from upset.

Errors VMG made:
Didn't define marker as an object, tried to define as a variable
Defined position incorrectly on happymap
I think the errors I made had to do with not understanding how these pieces are connected. Documentation is great at walking folks though steps if they already know what these pieces are. 

VMG Lessons Learned
Part of this learning process is how to read the documentation. Its logic is different than any help content I've read or used before.

Also, learning that best way to do this is by calmly tinkering.  I can't project how long these tasks will take me, and I can't shove them into time slots.  I just need to hack at it for periods of time, and not be surprised that it doesn't work.

Lesson 5: (10/24/2011)
From John: "Now you need to get all of your data with lat/lon coords and make a marker for each one. I think your goal had something about making shapes, so you should look at the Polygon documentation so you can learn how to add shapes. Why not try adding a triangle on the map?

You can make your home, work, and school the three corners.

http://code.google.com/apis/maps/documentation/javascript/overlays.html#Polygons"

Started working on Polygons tonight: http://jsfiddle.net/mozzadrella/YjkZX/4/
Not done in any way--this basically just copies and pastes the documentation
still need to figure out which pieces do what
And what arrays do and mean--because "array of arrays" just sounds like a 3-way mirror

From John:

"Remember that we talked about _types_?

234 is an integer
13.43 is a float
"abcd" is a string

[3, 4, 5] is an array of integers
[12.32, 43.25, 56.23] is an array of floats
["dog", "cat", "fish"] is an array of strings

[
  [3, 4, 5],
  [12.32, 43.25, 56.23],
  ["dog", "cat", "fish"],
]
is an array of arrays"

This exercise has prompted me to find out what Google Earth can do, and how we can use Geography instruction. 

Started looking into importing data into Google Earth pro--which totally does the political coloration I've been looking for--the user imports coordinates
http://www.youtube.com/watch?v=p9EgI_RbXBU&feature=relmfu

VMG needs to research--->Does Google Earth Pro have a feature to play data over time?  

Political lines=KML layers

Using drawn polygons
http://www.youtube.com/watch?v=feVa7DRgimk&feature=relmfu

Lesson 5 Map: http://jsfiddle.net/mozzadrella/YjkZX/8/

Lesson 6 From John:
We need to get lat, lon for all the points you want to put on the map. You should probably come up with a title for each point as well.

 From John 11-21-2011: Manually create an array, loop 
Ok so you need to make an array filled with all the data
of the points or shapes. I would start with markers
basically a single point (lat lon)
so you would have an array of arrays with two elements
or more correctly in JavaScript an array of Dictionaries with two elements each
{lat:'2342.342', lon:'23423.234'} is a dict
you can have an array of as many of those as you want
then you can use a LOOP

You should copy and paste all coordinates initially

yeah
https://developer.mozilla.org/en/A_re-introduction_to_JavaScript
read the part about for loops
basically you can do something like this
for (){
// do something
}
and it will do something n times
based on what you put in the ()
example, imagine we have an array called markers
filled with dicts 
  for (var i = 0; i < markers.length; i++) {
add_to_map(markers[i]);
}
this would call the function add_to_map for each dict in the array
assuming the function add_to_map takes a dict as a parameter
we could have markers contain 10 dicts
  or 10,000
it would be the same
ok
 for (var i = 0; i < markers.length; i++) {
12:51
for (var i = 0; i < markers.length; i++) {
the computer gets to that line
it creates a new variable called i
and sets it to 0
(it does this only once)
then it checks i < markers.length
if it's true
it does the code in the block (calls add_to_map)
if it's false, it does nothing and moves on to the next code after the closing }
if it called add_to_map
(ie it was true before)
it then does i++
which is the same as saying
i = i+1
++ is called the increment operator
which just means add one
— is called the decrement operator
(minus one)
 so you should try and make a loop
that does this
 and takes your manually created array of lat / lon
and calls the appropriate google maps function
you will get stuf
stuck
on the internal code of the for loop
because google maps wants you to give it lat and lon
not a dict
so you can do this
inside
instead of add_to_map(markers[i])
which means add the i-th element to the map
you can do this
markers[i].lat
which will give you the lat value of the dict stored in the i-th position of markers array
and you can do markers[i].lon
to get the longitude
because our dict looks like this
{lat:'423423', lon:'312.3124'}
 
Lesson 6: 11-27-2011
Operators: http://www.howtocreate.co.uk/tutorials/javascript/operators
Philly List of historical markers: http://en.wikipedia.org/wiki/List_of_Pennsylvania_state_historical_markers_in_Philadelphia_County

Loops: http://jsfiddle.net/mozzadrella/CBfNN/55/

Also, confused about arrays and objects--how are they different? 

LOOPS - some problems here
1 - undefined function "add_to_map"
2 - "new Array();"

var haus = new google.maps.LatLng(39.950787,-75.145565);
var mapOptions = {
    zoom: 15,
    center: haus,
    mapTypeId: google.maps.MapTypeId.ROADMAP
};
map = new google.maps.Map(document.getElementById("PhillyHist"), mapOptions);


var markers = new Array();
markers[0] = {lat:'39.94883', lon:'-75.14672'}; 
markers[1] = {lat:'39.9522', lon:'75.1446'}; 
markers[2] = {lat:'39.94692', lon:'75.14793'}; 
markers[3] = {lat:'39.96735', lon:'75.17315'}; 
markers[4]= {lat:'39.950787', lon:'75.145565'};

OR

var markers = [
    {lat:'39.94883', lon:'-75.14672'},
    {lat:'39.9522', lon:'75.1446'},
    {lat:'39.94692', lon:'75.14793'}, 
    {lat:'39.96735', lon:'75.17315'}, 
    {lat:'39.950787', lon:'75.145565'}
];

for (var i=0; i < markers.length; i++) {
add_to_map(markers[i].lat, add_to_map(markers[i].lon);
}

function add_to_map(lat, lon) {
    //do something here
}
       


From John:

You have one error in your code, and one in your data. Very close.

Here's how I diagnosed the problem:

I added "alert(i);" to the beginning of loop and clicked run. That makes a JavaScript popup box show the value of i during each step in the loop. "0" is displayed, but it doesn't pop up again as it should. Looking through the code in the loop I noticed that the JSON is invalid, you're referencing a variable called "PhillyHist" for the "title" property.

I think what you want is the string, "PhillyHist" which should be quoted, without quotes it means variable.

If you change just that one thing, it works.

You won't see your other markers, because they're not in Philly. They're placed at 75 degrees instead of -75 degrees (look at the difference between marker[0].lon and marker[1].lon and you'll see what I mean). 

VANESSA: HUZZAHHH!!! http://jsfiddle.net/CBfNN/112/

Next up you need to get a list of all your markers and feed it into that loop. You don't want to manually enter hundreds of coordinates, so we might try and find JSON that has all the info you need and load that. We could also look for an API... where are you going to source your data?