More on educating for cognitive/behavioral outcomes (not just content knowledge/skills)

“Human history,” said H.G. Wells, is “a race between education and catastrophe.”
(as quoted in the Washington Post blog)

Education, I think we can agree if we take the time to phrase it, is about more than knowing things - it is also about the way in which we think, and the behaviors that we exhibit through our lives.

One curriculum that seems interesting is the Tools of the Mind program coming out of Denver, and now being implemented in DC. It builds on Vygotsky's ideas about mental tools and is thoroughly modern and so on, developing self-regulation in young children. I think it's great, but I can't help noticing that it seems similar in practice to Montessori methods. Montessori uses a concept called "normalization":

'Normalization arises from concentration and focus on activity which serves the child's developmental needs, and is characterized by the ability to concentrate as well as "spontaneous discipline, continuous and happy work, social sentiments of help and sympathy for others."'

Whatever you say the theory behind it is, I think it's great that educators are thinking about these kinds of learning goals. I see things reported of the type "study shows certain type of activity improves cognitive abilities of students" and I think "why aren't all the schools having their kids do that?" I've recently started messing around with Lumosity, which is trying to be a sort of cognitive training program for people of any age. It's pretty fun - and possibly really cool for the brains of the world.

I often think of better thinking as a goal of math education - it's not just for math, of course. Nearby math, people need good scientific thinking. I liked this article about making science education more scientific:

'We need a more scientifically literate populace to address the global challenges that humanity now faces and that only science can explain and possibly mitigate, such as global warming, as well as to make wise decisions, informed by scientific understanding, about issues such as genetic modification.
...
'The particular intervention we have tried addresses student beliefs by explicitly discussing, for each topic covered, why this topic is worth learning, how it operates in the real world, why it makes sense, and how it connects to things the student already knows.
'No matter what happens in the relatively brief period students spend in the classroom, there is not enough time to develop the long-term memory structures required for subject mastery.  To ensure that the necessary extended effort is made, and that it is productive, requires carefully designed homework assignments, grading policies, and feedback.
'As a practical  matter,  in a university environment with large classes the most effective way for students to get the feedback that will make their study time more productive and develop their metacognitive skills is through peer collaboration.'

So there's some thinking about how to get people to really think scientifically. Less far afield than you'd think, here's what Ira Glass relates in explaining what makes a good story:

'The story has to have more in it than “here’s what they do.” They need to make up theories about the interviewees, Alex says, putting them in categories, comparing them with other things, attaching them to bigger ideas. They need to always be thinking “this is like this,” “this means that,” “this little thing is an example of this bigger thing.” Especially “this little thing is an example of this bigger thing.”'

That kind of cognitive stance in the world, analyzing experiences and events around us, understanding things more deeply and making connections, is a desirable outcome of a good education.

There is a free curriculum available online that tries to address some of these goals, called Connections: Investigating Reality. I don't know if it's perfect, but it is at least a bold attempt to do something really cool with education. I like this list they have, a sort of collection of design principles for the course:

• The future will be more complicated than the present. Old solutions won’t solve new problems.
• To solve problems, you need to make sense of the real world.
• In the real world, everything connects. You’ll need to understand “systems.”
• Because they’re the creators of all sciences and all arts, human societies are the most important systems you can study.
• Making sense of systems requires organized thought. School subjects aren’t very good organizers.
• Thinking about ways to organize thought improves how you do it.
• For sense-making purposes, the real, everyday world is a better “textbook” than textbooks about it.
• Everything you learn should be useful, right here, right now.
• Writing makes you think. (Keep a journal.)
• Dialog makes you think. (Work with others.)
• We’re not going to tell you much. We’re just going to give you a series of things to do and let you teach yourself how to make more sense of reality - yourself, others, the world

Learn-by-doing needs to be taken seriously, at least as a component of education, if not as the only component. There's a neat short TED talk about "studio schools" in the UK. Of course, not all activity is educational. And often the kind of activity that people do on their own is not what they need to do to learn new things. A sort-of-relevant quote from a piece about parent involvement that increases academic achievement:

'The kind of parental involvement matters, as well. “For example,” the PISA study noted, “on average, the score point difference in reading that is associated with parental involvement is largest when parents read a book with their child, when they talk about things they have done during the day, and when they tell stories to their children.” The score point difference is smallest when parental involvement takes the form of simply playing with their children.'

The above gives some education goals that are difficult to reduce to standardized test items. I don't think they are going to be particularly powerful without content knowledge and more basic skills (like adding, typing, reading, etc.) but neither will they necessarily develop if they are neglected. Let us educate well.

Free Will Resolved

The universe has helped me out considerably by having Sam Harris write this book called Free Will, which saves me time since now I don't have to write it but I can still direct people to a book that explains my views on the matter. Hurrah!

I don't think that it's perfectly written, but it isn't awful, and I think the average reader should be able to understand the meaning from it, which is something. Of course it's all correct. Perhaps my favorite quote:

"You are not controlling the storm, and you are not lost in it. You are the storm."

Summary of the whole argument: Doing what you want and therefore choose is not what we usually think of as free will, because you don't choose what you will want in the first place.

Let us go forth, living more freely, having thrown off the shackles of free will!

Excel 2007 PERCENTRANK is trash.

People and software do not always mean the same thing when they talk about percentiles, percent rank, and so on. Do not expect different software to give the same values. In particular, Excel uses a method that is probably not what you expect and does not correspond to methods implemented in scientific software.

For Excel 2007, in the case of getting a PERCENTRANK for a value that appears in the range, you will actually get (the number of items strictly less than the value) / (the total number of items minus one). This has the nice feature of (at least for distinct values) giving percent ranks that range from zero to one inclusive. It has the nasty feature of almost certainly not being what you thought it was going to be, and not being what you'll get from SAS, R, SPSS, SciPy, etc. (It is, however, mimicked fairly well in other spreadsheet software.)

It isn't immediately obvious how Excel works out the PERCENTRANK for values that don't appear in the range. Some sort of interpolation, certainly - but not one that was easy for me to guess quickly. I'd love to know what the heck it is.

And it isn't just that Excel is non-standard - it also appears to be buggy. Here's one bizarre example I came across of Excel 2007 at work, in which PERCENTRANK is not stable when values are multiplied (or divided) by 100, sometimes giving the same percent rank for different values, sometimes giving different percent rank for the same values. Check out the rows in bold. You should be able to replicate this in Excel 2007 if you like (with nine digits of precision requested from PERCENTRANK).

value	PERCENTRANK	value/100	PERCENTRANK
96.775	1	0.96775	1
93.6625	0.954545454	0.936625	0.954545454
93.3	0.909090909	0.933	0.909090909
93.0125	0.863636363	0.930125	0.863636363
92.7875	0.772727272	0.927875	0.818181818
92.7875	0.772727272	0.927875	0.772727272
92.475	0.727272727	0.92475	0.727272727
92.0625	0.681818181	0.920625	0.681818181
91.5	0.636363636	0.915	0.636363636
91.275	0.59090909	0.91275	0.59090909
91.0875	0.545454545	0.910875	0.545454545
90.9125	0.5	0.909125	0.5
90.9	0.454545454	0.909	0.454545454
90.8375	0.409090909	0.908375	0.409090909
90.2625	0.363636363	0.902625	0.363636363
89.425	0.318181818	0.89425	0.318181818
89.0625	0.272727272	0.890625	0.272727272
88.4375	0.227272727	0.884375	0.227272727
88.1	0.181818181	0.881	0.181818181
83.325	0.136363636	0.83325	0.136363636
82.3375	0.09090909	0.823375	0.09090909
78.15	0.045454545	0.7815	0.045454545
71.5125	0	0.715125	0

The moral of the story? DON'T USE EXCEL FOR ANYTHING, BUT ESPECIALLY NOT MATH.

Math for Good Thinking

I was listening to this Freakonomics podcast and they had Ellen Peters on. She said:

"Numeracy in general, what it should do, is it should help you to better understand information, first of all, and that kind of comprehension is sort of a basic building block of decisions across a variety of domains. But numeracy should also do other things. It should also help you just simply process the information more systematically. It should, in general, help you to get to better decisions that are more in line with the facts."

and I agreed, and felt hopeful about the world, but it was all a setup for her research conclusion that even or especially people with higher education have preexisting beliefs that are not affected by good thinking:

"Greater scientific literacy and numeracy were associated with greater cultural polarization: respondents predisposed by their values to dismiss climate change evidence became more dismissive, and those predisposed by their values to credit such evidence more concerned, as science literacy and numeracy increased."

This is concerning! I would like to believe that my math-teachers colleagues are making students into better thinkers. I am inclined to hope that hard science majors are less inclined to be biased by "values" than, say, MBAs, but everybody has been through high school and ought to be better than this.

I also recently read "Thinking, Fast and Slow" by Kahneman, which is wonderful. He talks about how he helped develop a textbook on rational thinking that was never adopted by Israel after they finally finished it. I would like to see a textbook like that. I think that could be the kind of textbook American schools could use. Do you know of any resources in this vein?

And this idea of our-subject-should-make-people-better-thinkers is not unique to math, of course. It's everywhere, and here's a relevant selection from a science point of view:

"We need a more scientifically literate populace to address the global challenges that humanity now faces and that only science can explain and possibly mitigate, such as global warming, as well as to make wise decisions, informed by scientific understanding, about issues such as genetic modification."

That's from an article series that makes the point that unfortunately, too often the result of science classes is not a really useful better understanding of science. Good stuff.

I think that cross-discipline work between math and ELA could be really valuable too, and is the sort of thing that should probably be done more. I'm thinking of things like analyzing the logical structure of an argument, looking at symbolic structures of logical fallacies, etc. I'm sure there are more good ideas in this realm. Perhaps what I mean is really better-structured, deeper, more cognitively demanding ELA.

I'm also thinking about this in relation to the Common Core hullabaloo, which I think is generally wonderful - who wouldn't be for deeper better standards that encourage real understanding of concepts? But then I see stuff like this: http://engageny.org/wp-content/uploads/2011/11/grade-4-handout.pdf which is supposed to be a *model* of Common Core-aligned work, and it kind of makes me retch. Am I wrong to retch? Is this an improvement on what's currently being done?

I hope the thread connecting all these things I've mentioned is evident to you! haha What are your thoughts? Are there really great math (etc.) curricula/texts being used now or being developed?

Ramblings on Learning Management Systems

This is seriously just very quickly edited text from an email I sent a while ago:



Date: 12 Dec 11 3:46 PM

First, consider two classes of skills or activities. One is "basic" skills that can be taught explicitly and practiced repetitively. I'm thinking of things like multiplying two-digit numbers, and I believe this class of skills can be taught and practiced really well with technology like Delta, Khan academy, etc. The second class are collaborative and "higher-level" skills that are best learned by working in teams on projects that have clear connections to the real world. I'm thinking of things like analyzing an issue, making a design, that kind of thing.

Now I grant that probably skills really lie on some continuous spectrum, but I think enough of the "basic" skills can be lumped together that the conceptual distinction can be useful.

To illustrate with math, problems with one correct answer are generally on the "basic" side and problems that admit of many solutions, like "design a system that uses data to evaluate NYC schools" or "engineer a device that solves this problem" are the other side. I feel like I want better names for these two categories. I propose "derivative" skills and "creative" skills. Derivative skills are skills you just learn from others, based on their work. Good. Also includes therefore knowledge of facts like "Mars is the next planet out in the solar system".

I think there is almost but not quite an analogy to pure and applied math. It falls apart because pure math of course has a creative leading edge, and applied math is about creatively using math to solve problems.

I think perhaps the pervasive corruption of debate on education is that some people seem to think that these two kinds of skill and education can succeed in isolation. And then you get kids with basic skills who can take a test but can't do anything creatively or solve problems in the real world, and you get kids who are creative but everything they create is complete crap because they don't have any supporting skills or knowledge.

I think that people who focus on derivative learning don't care much about the creative side because their kids do well on standardized tests. People focused on the creative side take one of two positions: they say either that basic skills are not important and shouldn't be tested, or that their kids learn them but somehow learn them differently through doing "rich" tasks.

And that last is my big gripe with some applications of constructivism, wherein educators set their students some rich task and expect it to take care of teaching the fundamental skills. I compare this to handing kids a chess board and expecting them to get good at chess just by messing with it. I'll have more to say about chess but first I'm curious if this all seems self-evident to you or if it seems like I'm missing something important. My claim here is that there are broadly two classes of important proficiencies, and that they might be best taught in two different ways.

**

Okay now the derivative skills form a foundation for further work, and I think it's fair to say that every student should have them. You should not have a few kids who know how to multiply and a few that always go to them to ask when they need something multiplied. (Never mind that this is effectively what a lot of managers do; I think we'd have a lot better managers if those folks had some skills too.) But this lack of even distribution of skills can easily happen with group work, as I'm sure you've seen. If somebody can already do it, I don't need to learn it. So I think that for many or all derivative skills, students should definitely demonstrate proficiency independently, and this will mean independent study and practice.

You can see that I'm getting toward learning management systems. What they already have is that they scale-ably (how is that spelled?) deliver individual interactive practice. They make attempts, to varying degrees, at delivering individual instruction (explanatory videos, etc.).

I think the as-yet unrealized great potential of technological solutions is to also deliver scale-able differentiation. They're the only possible solution I know of. One teacher can't work one-on-one with thirty kids at the same time. You're already doing much better than average because you can assign different kinds of problems on Delta to different students, but that still doesn't scale, because it is always more work for you to figure out what to assign and do so for every student individually.

What I envision is a system that incorporates some kind of machine learning with an internal representation of every student's current knowledge and skill set and customizes on the fly the experience that individual students have to optimize their progress toward educational targets.

What this means is: you don't tell students what exercises to do, and you don't tell the software what exercises to have the students do, you just sit the student down and tell the computer that the student needs to learn algebra, or whatever.

What would this be like? Well, it would be a little like Computer Adaptive Testing. If the student does something wrong, the computer makes an inference about what they know. If they do it right, the computer would make a different inference. So as an example, if you sit the student down to learn algebra and they already know it all, the computer will know that and tell you that the student is done. Bam. On to the next thing. But it isn't just Computer Adaptive Testing, it's Computer Adaptive Teaching, because based on what it infers about student knowledge the computer can offer different lessons, videos, activities, etc. to help move the student along.

This is a little bit like what School of One does; they have a huge library of computer (and classroom) lessons and assessments - but they spend a huge amount of time just deciding what each student should do the next day. Computer systems should be able to work this out more efficiently. Also, I think School of One has the wrong idea about the role and value of teachers. School of One makes sure to have teachers delivering lessons in the usual way, by talking and drawing on a board, so as to not "take anything away" from the teachers. But teachers shouldn't have to focus on these derivative skills. They should be facilitating the fun and interesting projects and other learning experiences, helping students with higher level skills. What they shouldn't have to do is worry that they can't get their kids to write a good essay because they don't know how to spell or capitalize properly.

What else is this like? It's a little bit like playing rated games against a chess computer. If I play and lose, my rating goes down and the next game I play is against the computer at a lower difficulty setting. This adjusting of difficulty makes it more fun for me (and there's a comparison to that idea of "flow" in matching difficulty to skill level) and by practicing I can get better at chess. But I would get better at chess more quickly if based on my play the computer also offered suggestions or lessons on how to play better in general. And this is possible.

So I imagine that we have a collection of lessons and a collection of assessments, and probably all or many of these are very short, like one question or a two-minute "lecture". (I'm not entirely sure that lessons and assessments really belong in separate bins; it might be better to have them all be together as "activities" or something to that effect.) And there's an internal space of concepts or proficiencies. This could be set up explicitly like "able to multiply six by seven" or "knows what thesaurus means" or "can find Atlanta on a map", or it could be completely generated by a machine learning algorithm and not necessarily correspond to our ideas of what the component skills are. The main point is that it will be more complicated than one value representing "skill in math" or something like that, the way your chess rating represents "skill in chess". But the system would learn based on user interactions that, for example, a student who can't do this also can't do that, or can do this but only after first doing that, and so on, so that a dynamic adaptive progression can be made for each student, and change as the student works.

I also think such a system would do well to incorporate ideas of carefully spaced repetition for maximizing retention in memory. Often a few repetitions at the right intervals are better than many repetitions in one bunch.

This is the outline of what I think could be a very good system for delivering real differentiated instruction in a way that scales across all students and can raise them all to very high levels of competency in basic skills. I think such an approach should be balanced with work on high-level rich tasks that build a whole suite of different skills, and I think it is in that realm that the importance of classroom teachers lies. In terms of time-share, I think it might be something like 30% individual work with technology, 70% collaborative project type work. (I think I'm not far off from what Khan has said on this.) (For one thing, the mental demands that this kind of computer work makes are very high. You can't just space out in the back of the room.)

I think that Delta and many similar projects are steps in the right direction toward enabling this kind of education, but I think there is a big missing piece and that piece is the kind of artificial intelligence approach that I outlined. So here's check-in number two: does it make sense, what I'm saying? Do you think it could be done? I think it would require a novel architecture, of course, but I think a lot of the components of Delta could fit into that architecture. What do you think would be the main hurdles of building something like this?

**

I think I sent you this link already: http://www.learningregistry.org/about because I think that project could also help form some of the content guts of a system like this. There is a lot of educational content out there, but solving the problem of getting that content to students at exactly the right time and evaluating whether they understand it is a difficult problem. It is a problem that I think could be solved by an approach like the one I'm describing. It looks like they have a name for systems of this type: a "learning management system".

added 2012.3.5: My best guess at what junyo is up to, with one of the founders of School of One, is that they're doing the big data machine learning data analysis buzzword xyz work toward some of the goals that I've mentioned for next-generation LMS.

Slang Equivalencies from Big L's "Ebonics"

This is just a cut down version of the lyric. Check out the song: Big L – Ebonics


weed smoke:lye
ki of coke:pie
lifted:high
new clothes on:fly
cars:whips
sneakers:kicks
money:chips
movies:flicks
crib:home
jacks:pay phone
cocaine:nose candy
cigarettes:bones
radio:box
razor blade:ox
fat diamonds:rocks
jakes:cops
got rubbed:got stuck
got shot:got bucked
got double-crossed:got fucked
bankroll:poke
choke hold:yoke
kite:note
con:okey-doke
got punched:got snuffed
to clean:to buff
bull scare:strong bluff
burglary:jook
wolf:crook
felony:F
got killed:got left
the dragon:bad breath
7:30:crazy
hit me on the hip:page me
angel dust:sherm
AIDS:the germ
chick gave you a disease:you got burned
max:relax
guns and pistols:gats
condoms:hats
critters:cracks
food:grub
victim:mark
sweat box:small club
ticker:heart
apartment:pad
old man:dad
studio:the lab
heated:mad
iron horse:train
champaign:bubbly
a deuce:a honey that's ugly
fine girl:dime
a suit:a fine
jewelry:shine
in love:blind
genuine:real
face card:hundred dollar bill
very hard, long stare:a grill
sneaking to go see a girl:creeping
smiling:cheesing
bleeding:leaking
begging:bumming
nutting:coming
taking orders:sonning
ounce of coke:onion
hotel:telly
cell phone:celly
jealous:jelly
food box:belly
to guerrilla:to use physical force
took a L:took a loss
show off:floss