Saturday, December 29, 2012

Highlights from "This Will Make You Smarter"

This Will Make You Smarter
organized by John Brockman

I previously read and enjoyed John Brockman's What We Believe But Cannot Prove, so when I saw this volume as I was in line at Barnes and Noble, I picked it up. This Will Make You Smarter is not as good. The question was this: "What scientific concept would improve everybody's cognitive toolkit?" It isn't a bad question. He was looking for people to give novel "shorthand abstractions" such as "market", "placebo", and "random sample" once were. Most of the contributors didn't seem to understand the question. Many entries were not interesting or novel in the least. There was a lot of hemming and hawing over how un-scientific the general public is. I thought the last quarter of the book was by far the best. Here are a few of my favorite topics.

Kakonomics (Gloria Origgi) "the weird preference for low-quality payoffs"
This gives a word to that phenomenon wherein people claim to require and provide high-quality work, but everybody tacitly agrees to accept and produce low-quality work because it's easier. To be found, at least in pockets, in education, government work, and perhaps any sufficiently large organization.

Kafabe (Eric Weinstein) "an altered reality of layered falsehoods, in which nothing can be assumed to be as it appears"
He doesn't provide a better definition, I think, but the example that makes it clear is professional wrestling, in which everybody acts as if there's real competition but in fact the whole thing is planned. Maybe kayfabe is a specific case of Robert Trivers' entry in WWBBCP ("deceit and self-deception play a big role in human problems"), and then perhaps kakonomics is a specific case of kayfabe. Oh what a tangled web we weave?

Aether (Richard Thaler) (suggested as a pejorative for black box terms in theories)
This provides a word for use in criticizing theories that don't actually explain anything. Some other entries refer to the danger, well described years ago by Feynman, of thinking that naming a thing is the same as understanding the thing, and this is a little bit similar, but not exactly. I particularly like that he criticizes economics, saying that (among other things) time-varying risk aversion is an aether.

The Einstellung effect (Evgeny Morozov) "trying to solve a problem by pursuing solutions that have worked for us in the past, instead of evaluating and addressing the new problem on its own terms"
I didn't know there was a name for this. I do this kind of thing all the time. For example, if I think of doing a new web project, I immediately jump to Google App Engine, because I've used it before - even though it seems to not have much of a future, has a raft of clear drawbacks, and in any event is certainly not guaranteed to be the best tool for the job. But once you build one tool, you want to use it again rather than building a new one.

QED moments (Bart Kosko) "know[ing] what proof feels like"
This isn't really new either, but I like his angle on teaching the epistemological nature of mathematical proof as compared to other ways of knowing and how this sort of reasoning fits into the human experience.

The Veeck Effect (Gregory Cochran) "adjust[ing] the standards of evidence in order to favor a preferred outcome"
Yet another good name for something you already sort of know about.

Everyday apophenia (David Pizarro) the tendency to "err in the direction of perceiving patterns where none actually exist"
As in superstitions like lucky socks, etc.

Now that I go through my notes a bit, maybe it's me who didn't understand the question. The entries I found most interesting were the ones that gave names to phenomena I had some familiarity with already, not entirely new phenomena. One of the goals of the book was to create more cognitive shorthand, after all. Note also that these terms are useful in this - as, indeed, there is utility in having a term with which to refer to a specific type of bird. And perhaps it is too much to ask for such a book to provide entirely new concepts. Certainly really important new ideas would find expression in other outlets before being published in a compendium of this type. So there is some value in this tome, although I do maintain that it is not nearly as much fun as What We Believe But Cannot Prove. Still a fun series; I think I may check out What Have You Changed Your Mind About? next.



Monday, November 12, 2012

Art is Art: Two Galleries


I went to see Edward Tufte's gallery, ET Modern, at 547 West 20th Street. He's a sort of demigod of data visualization. Now he does art such as the above, which announces that "ART IS ART / AND EVERYTHING ELSE / IS EVERYTHING ELSE".

To get to ET Modern, I walked by the Jack Shainman Gallery at 513 West 20th Street. It looked interesting. I went in. It was showing a collection of work by Hank Willis Thomas called "What goes without saying".


Those are hand-painted blow-ups of old button designs. I explored these and other works, and then I went on to ET Modern to see Tufte's aluminum Feynman diagrams.


Tufte has made quite a lot of these aluminum Feynman diagrams. Some of them are hung like this, which brought to mind the most memorable thing I think I've ever overheard at a modern art museum:
Now this, this would go well with my sofa.

I do like Feynman diagrams quite a lot, to be sure. After I had gone to the two galleries once, I went back to "What goes without saying" to check it out again.


The second time at this gallery, I happened on a group that was being led by the artist himself, Hank Willis Thomas. It was a group of students whose professor knew the artist. I joined the group and felt welcome enough. Mr. Thomas talked about his work, something about his intent, his process, what he thought was good and interesting about various things - it was very nice. His work is not necessarily suitable for hanging over couches at country clubs. His work is communicative. It is emotional. It is meaningful. I am afraid my photo does not capture it very well. His work bears close examination. It bears consideration and reflection. At the very least, it has something to say.

I went to ET Modern once more to get a last picture. As I was leaving I asked the man who sat at a desk by the door, selling Tufte's books and things from the kind of ersatz gift shop there, why there was a big red "emergency stop" button by the exit. His explanation was a bit awkward, but he did demonstrate to me that it was completely non-functional, a kind of cartoon joke on the wall. What he said was, "It stops customers who don't buy things here."

I did think some about Tufte's work. "ART IS ART /  AND EVERYTHING ELSE / IS EVERYTHING ELSE" - I was more than a little tempted to declare that Tufte's work falls clearly on one side of that division, and that Thomas' work falls on the other. But there is still a lot to like in Tufte's work, his books, even his art.


"IT'S MORE COMPLICATED THAN THAT"

Sunday, October 28, 2012

Success in Natural Language Processing is Human-Level Intelligence

There's a lot of talk about Natural Language Processing, NLP, using computers to deal with lots of text. The current state of the art is like cooking with only a colander and whatever ingredients fall out of the tree in your back yard. People are entirely too excited about a collection of weak-sauce results.

  • Sentiment analysis is a joke on the unpopular kids (brands) desperate to know if people like them. "How many times did they say the words Nike and Love in the same sentence? Huh? How many?" AKA let's-reduce-all-human-discourse-to-one-linear-scale.
  • More general word frequency analysis can be fun, just like the index of an arbitrarily long book. And you hit problems with grammatical changes right away, so you start using some clever stemming approach, and that either makes things better or worse, and the machine is sure as heck not going to know which it is.
  • Co-occurrence? That's the best you got?
  • Okay Google's machine translation is pretty cool, but Chinese Room is not real understanding or analysis.
Take a look at this public service ad on the NYC subway:


Here's what it says:

MTA
.info
What's next?
Poetry is back
in Motion.
Many of you felt parting was not such sweet sorrow.
So we're bringing poetry back in a very artful way.
Hopefully, you'll feel transported.
Improving, non-stop.

When I was in Korea and studying Korean, I tried to read and understand signs that I came across. If nothing else, I should be able to read the signs in the subway, right? Even if you speak English as a first language, if you don't regularly ride the subway in New York, you may not know what this sign is really saying. If you're clever, you can sort of guess, but probably not perfectly.

MTA is the Metropolitan Transportation Authority. This isn't stated in the ad, but humans can probably guess that this public-service-announcement-looking sign on the subway is probably from the subway people.

"What's next?" is a rhetorical question, and in fact a sort of MTA advertising series as they tell us what updates and changes are happening now or in the near future.

"Poetry is back in Motion" refers to the MTA "Poetry in Motion" series, a separate initiative that puts short poems in subway ad slots. Apparently they stopped doing this for a while, and now they're going to be back. Or maybe they've just failed to sell all the ad slots. Who knows.

Next our friends the MTA allude to Juliet's parting words to Romeo from her balcony. Apparently by the end of the play both Poetry in Motion and all subway riders will be dead. But seriously, this allusion just doesn't make sense. Does no one understand the feeling Juliet was conveying? Honestly?

The "feel transported" is actually kind of a fun double-meaning pun. The "non-stop" is less fun but would probably be better if it wasn't following a bunch of other junk just like it.

I suppose you could say that the MTA has really done a noble job in making a dull message a little more fun. No doubt. The point is that really understanding even this fairly simple message is not so easy. What if your NLP doesn't have the NYC-subway-rider plug-in? Or the dual-meaning-pun plug-in? The Shakespeare plug-in? I suspect that until machine text analysis is done by an embodied learning computer with human-equivalent intelligence, we will be limited to the frankly unimpressive kinds of tools that we have so far. To make my suggestion even less helpful, I suspect that as soon as such technologies exist, they will have the same drawbacks that humans do. Perhaps computer users will all have to be managers. Will I have to give my computer the weekend off? Maybe I should have... ramble mode OFF

Saturday, October 27, 2012

metric-driven vs. data-driven


Bitsy Bentley, director of data visualization at GfK Custom Research, gave a good talk on Monday at Pivotal. It was mostly about visualization, but the part that resonated most with me was a related point she made about the difference between being metric-driven vs.data-driven. Here's how she illustrated where most groups currently are and the direction she thinks they need to move:

Inline image 1

Some people in the audience on Monday were confused about the distinction between metrics and data. I think it's an absolutely vital distinction, and one that hits close to home when I think about work that I'm often asked to do. A metric is a particular reduction from some subset of your data. It can be reported, rewarded, punished, used for other decisions... Metrics can certainly have value, but being focused just on some metrics is not the same thing as being data-driven. The stories in data, the real information, they frequently resist reduction to metrics - certainly to the limited collection of metrics you happen to already have. And metrics frequently obscure rather than elucidate. At best they give you a rough what - rarely a useful why or how.

As a hypothetical example, you might feel good watching a metric march in the right direction for a number of years - but if it does move in the wrong direction, that metric doesn't tell you why or what to do about it. If it was evidence of success before, is it evidence of failure now? What if you aren't doing anything differently? To really make decisions based on data requires more than just monitoring metrics. And I don't just mean you need the right metrics rather than the wrong ones - metrics are necessarily reductive, and even if you have the best metric perspectives on your data, they are still perspectives, not the data itself.

One conclusion is that it's often better to plot all of your data, as much as possible, to have a chance at understanding it before you start reducing it to numeric summaries. A corollary might be that we should question metrics that don't show the whole picture. Another conclusion is that we need to spend more time dealing with the data itself in order to understand its nature, to identify which metrics might aid understanding and which effectively stymie it, to determine what is signal and what is noise, and vitally to spend more time looking for new things than we spend recreating old things and then quickly close the loop by acting on new insight (reporting, changing policy, etc.) - and then go back to looking for the next thing.

I think this could be something to think about: are we data-driven, or are we merely metric-driven?

Ponder the divine wisdom of data cat:

Inline image 2

Sunday, August 12, 2012

Exploring Everyday Things with R and Ruby

Exploring Everyday Things with R and Ruby
Sau Sheong Chang

This book came out recently. Somebody had suggested it might be fun, so I read it. Sure enough, it is fun. It introduces a lot of really good stuff, perhaps briefly and imperfectly (and you will read "lot" instead of "plot" in at least one place) but it conveys a sense of wonder and possibility. It reminded me of the books of science experiments that I grew up with. This book is structured a bit like they were, with about eight investigations into various things. It guides you through building a digital stethoscope and processing the data it produces - and then it does the same for using a digital camera to take your pulse by reading differences in red intensity as a result of varying oxygen concentrations in your blood. It's really quite neat.


The focus is not really on building physical objects though - it's on the computer side, for simulation and analysis. The techniques weren't really new to me - and in fact in one place the author spends nearly a full page explaining the Pythagorean Theorem - but the spirit of boldly applying techniques to interesting problems is a good one. I would feel pretty good about recommending this book to a middle or high school student with an interest in technology, and any others with curiosity. To be fair, there were good pointers to things I wasn't intimately familiar with, like the Shapiro-Wilk normality test, and while not novel or very deep, the introduction to and work with ggplot2 in R is definitely widely applicable. I'm still not sure I like Ruby more than Python, but you can quickly get a feel for doing things in Ruby as well. It's a fun little book for getting you thinking, and then hopefully looking for more information and working on your own experiments.

Sunday, August 5, 2012

Memorization vs. Understanding is really Breadth vs. Depth

It is more difficult to think and communicate about things without names. Almost always, when learning something it is helpful to attach useful language. Importantly, by far the more time-consuming task is the understanding of the concept, not the learning of the term. For any given learning then, there is no real advantage to avoiding the details of terminology.

For example, students could in theory learn about evolution without learning the term "evolution" - but it would not be a better way to learn. Students could learn about a president's decisions without learning the president's name - but again, this would not be an improvement.

It is possible to memorize terms without fully understanding underlying concepts, and even if this allows a student to pass a poorly-designed exam, it is clearly not an admirable learning goal. If there are a very large number of things to learn, however, it may seem that there is only time for learning their names.

Memorizing terms alone is not a complete education. But the apparent alternative, somehow learning without learning terms or details of place and time, is not a strong alternative. We should embrace the language that accompanies learning. It may be that we need to focus on a smaller corpus to allow time for deep understanding, but that understanding will not live long without the words to discuss it.

Wednesday, June 27, 2012

Doubles are sufficient for all data relations

I was interested a while ago in RDF, which stores information in triples, like "the sky" -> "has the color" -> "blue". I think there's too much structure in the triples structure. This can all be reduced to doubles, or just a simple directed graph, through judicious use of blank nodes. So the example becomes "the sky" -> blank node that could be interpreted as having-a-property, that blank node -> "has the color", that blank node -> "blue". (If you want you could add "has the color" -> "relation" and "blue" -> "property" or similar.)

This is of course not necessarily the most efficient way to store information, or the easiest to work with, but it reduces the structure so that all the primitives are of the same type, and I don't think that the structure can be simplified any further, so in that sense it represents a kind of absolute in data structuring.

I don't know what this kind of data structure is called by others or if there is any existing work around it. I would like to know!

Thursday, June 7, 2012

How hiring works now

The problem with the job market is not that there aren't enough jobs, it's that the jobs there are can't be done by enough people. What unites them is not even that they are technology jobs; many are visual design jobs that happen to use technology. I think what really unites them is that they are habits-of-mind jobs. They are jobs for people who spontaneously do and make things - not to fill up a resume, but because they are naturally active, productive people.

Anyway, a lot of people are hiring these days. Here's a bit from a notice I saw on an email list, with my comments:

1. Do not send us a resume. Please. Don't. We won't read it.
I'm so glad other people are starting to agree with me about this. Resumes are awful.


2. Email us at ////// at /////////////.
I remember a time when teachers told me I had to go buy resume paper to print my resume on. Now most people are understanding that email is how people communicate. I'd like for companies to be required to disclose whether they have a fax machine - so I know which companies not to invest in.

3. Tell us about yourself; your hopes, dreams, desires or, better yet, how you like to code.
I like all of these, but especially the response to that last bit would tell you a lot about whether a person actually knows anything.

4. Send us examples or your work: code snippets, urls, github, etc.
This is the main point, I think: What matters is what you've done. I don't care if you aced every course at Harvard and MIT and have a recommendation from Obama. Show me what you do. I don't know that portfolios are always appropriate at every stage in education, especially because it's at least as bad as resume-focus when people focus on portfolio-stuffing.

5. Be prepared to Skype with us!
Duh.

Tuesday, June 5, 2012

Why I don't like vocational education

Education should improve people. Training to do particular things can be a part of that, but it isn't the whole thing. Vocational schools say, in effect: "You are a lost cause. We can't help you get any better as a person, but maybe we can train you to work a particular job at a business that may or may not exist by the time you graduate." Vocational schools represent an abdication of responsibility, an abandonment of education's central commitment to humanity. They reduce their students to machines. Schools can certainly have vocational components - but a vocational school barely deserves to be called a school at all.

Monday, June 4, 2012

Expected value is not useful for making decisions about lottery tickets

It was a surprise to me when I learned that sometimes lottery tickets actually have positive expected values. This happens only rarely, and generally only for progressive jackpots. But I was alarmed that it happens at all. So often in the literature people are judged as irrational if they don't act in accordance with expected value. Should I buy a lottery ticket with a positive expected value?

For example, say someone offers you the following bet: we flip a coin, and if it comes up heads you get $10. If it comes up tails you lose $1. This game has expected value of +$10/2 + -$1/2 = +$4.50. Should you play this game?

If I can play this game arbitrarily many times, and I have a handful of dollars to start out with, I will play this game as a full time job, and I will be rich. After playing just six times the odds of losing money overall are under 1%, and that number goes even further to zero the more I play. I am quickly in a world where it is more likely that I am better off than I was before.

However, invite me to play this game just once, and I am ambivalent. Maybe I'll make some money, but it is equally likely that I will lose money. I don't think it is irrational to decline the offer to play just once. (Some might say that it is irrational because it should become one of many positive expected value opportunities that I take, ensuring a positive outcome overall, but (a) positive expected value gambling opportunities are notoriously rare, and (b) the typical slimness of the odds matters, as follows.)

Even the best positive expectation lottery ticket has odds much worse than one in a million, so let's take that as the odds for a $1 ticket. Say the prize for this bet is $2 million. The a ticket has an expected value of nearly +$1. (For simplicity in the following, I treat each lottery ticket as independent, which is not quite true, but is not such an awful simplification since in reality you can't buy enough lottery tickets to benefit significantly by owning a large portion.)

In the coin flip bet above, if you play twice you have over a 50% chance of having more money than you started with. How many times do you have to play this excellent one-in-a-million lottery game to be more likely to have won money than lost? log(0.5)/log(1 - 1/1,000,000) = nearly 700,000 times.

Both the coin flip and lottery here have positive expected value, and if you could play them arbitrarily many times, both would be cash cows. In the case of the coin flip, the generous single-game odds make it fairly easy to ensure that your odds of winning overall are excellent. But as the odds get worse, as in the case of the one-in-a-million bet, (still not bad, by lottery standards) the nature of the physical universe makes it nearly impossible to play enough games to achieve decent odds of a positive outcome.

I think it is rational to consider the probability of the overall outcome of gambling rather than the expected value of the gamble. It is not just about psychological "risk aversion" and it is not just about ignorance of expected value.

I have never seen an army of hedge fund quants scrambling for the nearest 7-11 to invest heavily in high-jackpot Powerball tickets. In any event, I won't be buying any - regardless of expected value.

Saturday, June 2, 2012

Thoughts and selections from The human side of school change

The human side of school change: Reform, resistance, and the real-life problems of innovations
Robert Evans


Here is a consultant who knew in 1996 that he needed to start saying "innovation" a lot. So now he's the keynote speaker at the upcoming NYC principals' conference, and all the principals are getting a copy of his book.

There is a lot to like about the book. The typesetting is gorgeous, for one. I wish it said what the font face is. But the author really does seem to be very intelligent, and the logical organization of the book is pretty good. It does irk me a bit how he seems to misuse "axiom" and to cite sources for claims that appear to be purely opinion. Overall: it's a book. There are some interesting bits:


One thing that I find interesting is how similar advice for principals managing teachers is to advice for teachers managing students:

"The extensive history of school reform has offered researchers ample opportunity to study the training that supports innovation. The findings are straightforward: to help teachers develop new competence, training must be coherent, personal, and continuous. Coherent refers to the design and sequencing of the training content. Sessions must be relevant to the innovation and unfold in a logical way that provides teachers with both an overview of the larger goals and a walk-through of the specific objectives and the methods for achieving them. Personal refers to the importance of tailoring training to the current knowledge, practice, and felt needs of teachers. ... New practices depend crucially on the amount of personal assistance teachers receive and how user-friendly this assistance is." (p. 63-64)

The continuous bit is about not expecting training once at the beginning to be sufficient forever onward. This is all good stuff, but hardly different than what works for teaching students. This comparison across organizational levels is an interesting perspective only occasionally encouraged in the text.


"Schlechty (1990, pp. 7, 17-28) identifies three competing conceptions of the purpose of schools that have influenced restructuring: as a tribal center (to transmit cultural enlightenment and traditions and norms of citizenship), as a factory (to select and sort students for future occupations according to standardized measures of performance), and as a hospital (to redress the pain and suffering imposed on children by society and to equalize opportunity)." (p. 89)


"(One of the most draining aspects of change for a leader is the seemingly endless need to correct misimpressions, to answer the same questions yet again, long after she had thought everyone understood.)" (p. 77)


'With regard to heterogeneity, despite academic research that tends to support its advantages for most students, moral arguments made for it on grounds of equity and social responsibility, and periodic reports from schools of remarkable successes, teachers continue to oppose it overwhelmingly. For one thing, heterogeneous classes are harder to teach than homogeneous classes of equal size. Since few districts cut class size when they change grouping patterns, heterogeneity makes teachers' lives more difficult. Moreover, the range of heterogeneity is expanding as the student population becomes more diverse and as inclusion introduces into classrooms greater numbers of special needs students with major learning and behavioral problems. I vividly recall a biology teacher in an urban high school describing what restructuring meant to him: "Over the summer, our high school was detracked. Just like that. In each of my five classes, I still have thirty-three kids, but they now speak eight different languages, their reading levels go from the 97th percentile down to the third, and there are at least four or five - in every class, mind you - with serious behavior or emotional problems. I'm almost totally reduced to multiple-choice worksheets. Some days I'm furious; others I'm depressed."' (p. 82)


It's a little interesting the way the author seems to sometimes refer to students as clients.


"Implementation happens during the school year and after the school day - there is no closing the factory to retrain the workers." (p. 85)

More and more, I think teachers should work all year. Even if students still get a summer vacation, teachers could develop curricula and so on during the summer. I remember how shocked I was after getting my first teaching job, at a public high school. "Do we have any sort of planning, workshops, or anything like that during the summer?" "Nope, just show up the day before classes start." That's no way to run a school.


"Restructuring exaggerates the causal role and curative reach of the school." (p. 86)

Probably true, not necessarily helpful.


This bumper sticker is quoted: "I Feel Better Now That I've Given Up Hope" (p. 95)


"It is an axiom of career theory that occupational satisfaction and achievement require require congruence between the characteristics of the individual and those of the work environment."

I tend to agree that that congruence is important, but I also think the author is here abusing the term "axiom" and exaggerating the degree to which "career theory" is the kind of thing that has axioms anyway.


"Schools have always depended for their success on a high degree of exploitation - getting maximum effort for minimum compensation - and it is far easier to exploit eager, innocent, unencumbered beginners than weary, wily, overloaded veterans!" (p. 117-118)


After listing these pressures of teaching: social complexity, multiplicity (which we might call multi-tasking), personal involvement, motivational burden, public nature, unpredictability, and professional isolation: "Every profession, of course, has its own pressures, but these characteristics make teaching an unusually draining activity, one marked by a sharp disparity between giving and getting." (p. 121)


"The deprivation they experience, coupled with many of the life and career changes noted in Chapter Six, incline teachers more strongly toward what is sometimes called a "union mentality" - that is, a militant antimanagement posture, a to-the-minute definition of the work day, and a reflexive, legalistic opposition to virtually any innovation that might impinge on contractual agreements. This may not be an inevitable response to a sense of being taken advantage of, but it is, in my view, a natural one. Employees who have long felt underpaid and who then see their social status slipping even as demands upon them increase are more likely to adopt an agressive, us-versus-them posture. The irony is that while this can increase teachers' muscle, it further diminishes their status. When it leads them to oppose changes in working conditions necessitated by certain reforms (say, a reduction in individual preparation periods to create more common planning time or a change in certification requirements to permit more interdisciplinary courses), it gives ammunition to their critics and leads the general public to see them as not only unprofessional but opposed to improvement." (p. 124-125) (emphasis mine)


"... replacing a traditional credit-hours graduation requirement with a formal exhibition of competence" (p. 126)

I just think this is a neat idea.


"... the year the Russians launched Sputnik, the first space rocket, terrifying America ..." (p.131)

I have difficulty taking an author seriously who describes Sputnik as "the first space rocket."


On page 136 the consultant author includes consulting as one of three major costs facing schools that want to succeed in restructuring. This guy is a consultant.


(quoting Bennis and Nanus, p. 21) "Managers, it is said, do things right, while leaders do the right thing." (p. 148)


(quoting Kouzes and Posner, p. 27) 'In this regard, the distinction between management and leadership is that managers "get other people to do, but leaders get others people to want to do."' (p. 172)


(quoting Kouzes and Posner, p. 43-44) 'Rather than "What gets rewarded gets done," we should apply a new axiom, one that helps explain why people seek to excel: "What is rewarding gets done."' (p. 172)

I don't know who it is for sure here, but somebody has problems with what "axiom" actually means. I like the idea though.


'The primary leadership strategy here is "what is good gets done."' (p. 173)


I do like the author's focus on good leaders acting in consistence with their values and otherwise being "authentic."


'An excellent example is Carl Glickman's call for schools to adopt a single overarching goal: to prepare students to be productive citizens of a democracy. Besting other nations in math or science, teaching basic skills or critical thinking - these are "subgoals of the larger, single goal of public education. When these subgoals are treated as primary goals, they lead to fragmentation, vulnerability, and despair as schools try to be all things to all people" (1993, p. 8).' (p. 228)

I like this a lot.


"What does focus mean in practice? It means something as elementary as it is rare: pursuing one major change at a time per person and per work group." (p. 217)


'Teachers' relations with one another are mostly marked by congeniality (being pleasant) but not collegiality (serious professional interaction). Though collegiality's benefits are "obvious, logical, and compelling," it is "the least common form of relationship among adults in schools," observes Barth (1989, pp. 229-230). ... The entrenched norms that prevail among teachers remain those of autonomy and privacy, "not open exchange, cooperation, and growth" (Johnson, 1990, p. 179).' (p. 233-234)


"Whatever they promise, shared governance and collaboration always mean more work - and more complex work, and more work with other adults rather than with students. They require greater investment in the workplace and higher levels of sophisticated adult interaction. Many teachers welcome neither." (p. 234)


"There is also the truth that many teachers, regardless of age, prefer to work with children than with adults. ... Even faculty who are gifted with students can be unhappy and awkward with adults. Two of the best teachers my children ever had were fabulous to watch in a classroom but incapable of looking a parent in the eye during a conference." (p. 234-235)


'This means that technical communication among teachers is more difficult, less necessary, in some ways even less appropriate than it might seem. It is moe difficult because two people can teach the same curriculum to similar students but operate in vastly different ways on vastly different assumptions that are hard to explain, let alone bridge. It is less necessary because in the most basic practical terms, schools can easily function as a set of independent workshops (quite unlike hospitals, for example, which literally cannot operate without close linkage among staff). And it is less appropriate because the separateness and "professional egalitarianism" that incline teachers to keep to themselves is routine among artisans. "Noninterference with the core work of others constitutes a sign of professional respect," while asking for assistance can seem a sign of weakness and offering unsolicited help a sign of arrogance.' (p. 235)


"From a school perspective, the competitive corporate ethos can seem cruelly hard. From a business perspective, the school ethos seems childishly soft." (p. 275)


"But changing education, Elmore and McLaughlin suggest, is rather like changing a language: formal efforts to modify usage don't usually succeed at first and rarely turn out exactly as they were intended, but over time a language makes dramatic changes, incorporating a variety of new influences in diffuse and uneven ways." (p. 294)


'Without a basic "values consensus" about the purpose of education, substantial improvement in school performance is unlikely (Schlechty, 1990, p. 28). This is not a call for national curriculum standards but for a better purposing at a fundamental level.' (p. 296)

Thursday, May 31, 2012

Wrapping up Five Minds for the Future

Five minds for the future
by Howard Gardner (yes the multiple intelligences guy)



I started this book, stopped, wrote up some interesting selections in another post, then finished reading the book. It's an okay book. I think I initially got it because I had Gardner confused with someone I like more. But Gardner's not bad. I like a lot of the things he has to say in this book.



Here are selections that I comment on:


"Teachers ought to illustrate the several ways in which a particular math problem can properly be solved or a literary passage can be interpreted; ..."

Maybe this is an important difference between math and literature. In math, there can be many ways to solve a problem, but after a solution is reached it is either correct or incorrect. Of course there are cases where perhaps only one of several solutions is found, and then this all does get more complicated at advanced levels when we are concerned with proving theorems and, more to the point, first finding theorems to prove - but correctness is in some sense fundamentally verifiable. In literature, it seems the focus is often on multiple equally viable interpretations. Perhaps I am wrong here, but at the very least it does seem that the experience of reading and benefiting from literature varies much more and depends much more on uniquely personal interactions between reader and text in ways that are different from the experience and benefits of doing math.


"A truly respectful individual offers the benefit of the doubt to all human beings. As much as possible, she avoids thinking in group terms. She reserves censure for those who truly deserve it. She remains open to the possibility that her judgment may have been wrong. And she is on the alert for a change in behavior that will in turn reinstate a feeling of respect toward that other individual.

"In my view, respect should not entail a complete suspension of judgement. When a person consistently acts disrespectfully toward others, that person should be called to account. And should disrespect persist, and deteriorate into frankly antisocial behavior, that person should be ostracized from society. (On rare occasions, an entire group may forfeit its right to be respected.) Mahatma Gandhi kept reaching out to Hitler; the Indian leader wrote a letter to Hitler, addressed Dear friend,' calling on him to change his tactics and promising him forgiveness in return. In turn, Hitler remarked, 'Shoot Gandhi, and if that does not suffice to reduce them to submission, shoot a dozen leading members of Congress [Gandhi's political party].' When unconditional respect inadvertently encourages antihuman responses, it is counterproductive."

This is interesting for two reasons. First, I had never heard that story of Gandhi and Hitler, and it's an interesting story. Second, this is one of several places where Gardner recommends ostracizing some people. Maybe this is the only reasonable choice when faced with true sociopaths - but it is still troublesome, seemingly inconsistent with virtue, and certainly not an option open to, for example, a teacher in a classroom faced with a sociopath student.


"Particularly valuable evidence comes from studies of rescuers - inhabitants of Nazi-occupied Europe who, at considerable risk to themselves, elected to hid Jews or other hunted individuals. According to Samuel Oliner, rescuers appeared quite ordinary on the surface; they resembled many others who were bystanders and even some who actively aided the Gestapo. Closer study revealed telltale differences. Rescuers were marked by a childhood during which their parents avoided physical punishment, opting instead for lucid explanations of rules and practices. The rescuers stood out from their fellow citizens in the strong values - often but not invariably religious - that they absorbed from their parents; a constructive and optimistic stance that they assumed toward life; feelings of connectedness to others, even those from a different group; and above all, an intuitive (indeed instinctive) reaction that what was being done to the innocent was wrong and that they themselves were capable agents who ought to (indeed, who must) take corrective action.

This is really interesting. I think it speaks to the importance of ideas about the nature of the universe that are formed (and perhaps largely influenced by external environment rather than genetics) at an early age based on the family setting.


"In what kind of a world would we like to live if we knew neither our standing nor our resources in advance?"

I really liked this question, used in introducing ethics.



Here are selections that I like, without comment:


"Syntheses are put forth all the time - for example, most textbooks and many trade books (including this one!) are frank efforts to synthesize knowledge about a possibly unwieldy topic so that it can be assimilated by a target audience. Determining what constitutes an adequate synthesis is not possible; as with the proverbial question "Does a string stretch across a room?" the answer must be contextualized. It turns out that arriving at an adequate synthesis is challenging, and anticipating the criteria for a judgement even more so."


"Individuals differ significantly in their predisposition to metaphorize, and in their capacity or inclination to transfer lessons from one class or discipline to another. Aristotle deemed the capacity to create apt metaphors as a sign of genius."


"Secondary-school students cannot be expected to be scientific or historical disciplinarians."


"Until recently, creativity has been seen by psychologists as a trait of certain individuals; as such it should be measurable through paper-and-pencil tests; and an individual deemed "creative" should be able to evince that trait across various performance domains. In the prototypical item on a creativity test, subjects are asked to think of as many uses as possible for a paper clip, or to give an imaginative title to a squiggle, or to choose the target that can be associated with two supplied words (mouse-cottage: both can be linked to cheese). The final tally received on such a measure is believed to reflect creative potential in any domain of knowledge."


"A wit said of Camille Saint-Saëns, an aging musical prodigy who never fully realized his early promise: 'He has everything but he lacks inexperience.'

"The creator stands out in terms of temperament, personality, and stance. She is perennially dissatisfied with current work, current standards, current questions, current answers. She strikes out in unfamiliar directions and enjoys - or at least accepts - being different from the pack. When an anomaly arises (an unfamiliar musical chord, an unexpected experimental result, a spike or dip in the sale of goods in an unfamiliar territory), she does not shrink from that unexpected wrinkle: indeed, she wants to understand it and to determine whether it constitutes a trivial error, an unrepeatable fluke, or an important but hitherto unknown truth. She is tough skinned and robust. There is a reason why so many famous creators hated or dropped out of school - they did not like marching to someone else's tune (and, in turn, the authorities disliked their idiosyncratic marching patterns).

"All of us fail, and - because they are bold and ambitious - creators fail the most frequently and, often, the most dramatically. Only a person who is willing to pick herself up and 'try and try again' is likely to forge creative achievements. And even when an achievement has been endorsed by the field, the prototypical creator rarely rests on her laurels; instead, she proceeds along a new, untested path, fully ready to risk failure time and again in return for the opportunity to make another, different mark. Creative activity harbors more than its share of heartaches; but the "flow" that accompanies a fresh insight, a break-through work, or a genuine invention can be addictive."


"Executives realize at a deep level that creativity is a chancy undertaking that can never be guaranteed- only fostered or thwarted."


"Undisciplined creativity is creativity undermined."


"Turning to specific disciplines, I do not believe that science and mathematics ought to be inflected as a means of honoring group differences. As universal languages, these ought to be construed and taught similarly around the globe."


"In the long run, rule by fist, fiat, fear, and fury is destined to fail."


"'A person who is determined to do something constructive with his life needs to come to terms with the fact that not everyone is going to love him." (Barenboim)


"They asked, in effect, what kinds of citizens do we want to produce?"


"Perhaps, indeed, there are no truly universal ethics: or to put it more precisely, the ways in which ethical principles are interpreted will inevitably differ across cultures and eras. Yet, these differences arise chiefly at the margins. All known societies embrace the virtues of truthfulness, integrity, loyalty, fairness; none explicitly endorses falsehood, dishonesty, disloyalty, gross inequity."


"An education centered on good work

"Until the third decade of life, young persons spend more time in school than in any other institution. They are in the presence of teachers more than in the company of parents; they are surrounded by schoolmates more than by siblings or children in the neighborhood. Formal educational institutions play a key role in determining whether an individual is proceeding on the road to good work and active citizenship."


(endnote) "Benjamin Bloom, Taxonomy of Educational Objectives (New York: Long-mans, Green, and Co., 1956)."


"(An aside on literacy: the first cognitive assignment for all schools is mastery of the basic literacies of reading, writing, and calculation. Because this point is an has long been uncontroversial, I need not elaborate on it here.)"


"In any event, creativity goes hand in glove with disciplinary thinking. In the absence of relevant disciplines, it is not possible to be genuinely creative."

Sunday, May 27, 2012

Indirect arithmetic for more engaging practice

This article is interesting in and of itself, on the psychology and reality of cheating, but I also very much like the task they had people do in some of the experiments.

In this grid, find the two numbers that add to ten.

Problems like this are very different from typical arithmetic exercises like "what is 5.82 + 6.36?" For one thing, it is a bit more of a puzzle rather than a question, which makes it seem more fun to me. For another, it rewards highly skills like estimation and working logically to break down the problem. In this example, you might first see that the 1.69 and 1.82 can't possibly be involved, since there is no other number greater than 8. Then one might start looking for reasonable pairs with final digits that add to ten, and so on. Compared to a more traditional approach of the form "estimate 5.82 + 6.36" which seems pointless because it lacks a context that makes the estimating appropriate, problems of this grid type provide a real context, even if only in the context of the game.

I have seen similar examples, such as the card games I played with middle school students in an after school math program, in which students are never asked to do arithmetic for its own sake, but are asked to play games that require doing arithmetic for success. It makes for much more engaging practice.

Wednesday, May 23, 2012

Lectures vs. Videos

Can a video replace a lecture? I'm not sure.


I go to lectures I would never watch a video of because a lecture can be an enjoyable experience. It's an event. You go, you sit, you meet people, you experience the whole thing with no scrubbing through to find a more interesting bit. It may be more memorable because of this immersion in a novel environment, because it can encourage more engagement.

I think research could shed light on whether video can be as effective as a non-interactive lecture. Test students after a non-interactive live lecture that is recorded. Have other students watch the recording and test them. There may be some positive effect of being live. Maybe some research like this has been done - I'd like to see it.


Lectures also happen to me spontaneously when someone figures out that I would benefit from hearing something at a particular moment. Teachers are often good at figuring out what you need to hear.

I think there is some room for computer adaptive systems to get better at figuring out what you need to hear and when. I don't know that they'll be as good as a good teacher any time soon, but they may be better than a bad teacher rather quickly.

Tuesday, May 22, 2012

Quotes from Rework

Rework
by Jason Fried and David Heinemeier Hansson

This book is largely the same as Getting Real, which you can read for free online. Both are pretty good, and pretty quotable. Here are some quotes I like, from Rework:


"Instead of describing what something looks like, draw it. Instead of explaining what something sounds like, hum it. Do everything you can to remove layers of abstraction.

"The problem with abstractions (like reports and documents) is that they create illusions of agreement. A hundred people can read the same words, but in their heads, they're imagining a hundred different things."


"Maybe it's because of the copy-and-paste world we live in these days. You can steal someone's words, images, or code instantly. And that means it's tempting to try to build a business by being a copycat.

"That's a formula for failure, though. The problem with this sort of copying is it skips understanding - and understanding is how you grow. You have to understand why something works or why something is the way it is. When you just copy and paste, you miss that. You just repurpose the last layer instead of understanding all the layers underneath.

"So much of the work an original creator puts into something is invisible. It's buried beneath the surface."


"Businesses are usually paranoid and secretive. They think they have proprietary this and competitive advantage that. Maybe a rare few do, but most don't. And those that don't should stop acting like those that do. Don't be afraid of sharing."


"There are plenty of companies out there who have educational requirements. They'll only hire people with a college degree (sometimes in a specific field) or an advanced degree or a certain GPA or certification of some sort or some other requirement.

"Come on. There are plenty of intelligent people who don't excel in the classroom. Don't fall into the trap of thinking you need someone from one of the "best" schools in order to get results. Ninety percent of CEOs currently heading the top five hundred American companies did not receive undergraduate degrees from Ivy League colleges. In fact, more received their undergraduate degrees from the University of Wisconsin than from Harvard (the most heavily represented Ivy school, with nine CEOs).

"Too much time in academia can actually do you harm. Take writing, for example. When you get out of school, you have to unlearn so much of the way they teach you to write there. Some of the misguided lessons you learn in academia:

  • "The longer a document is, the more it matters.
  • "Stiff, formal tone is better than being conversational.
  • "Using big words is impressive.
  • "You need to write a certain number of words or pages to make a point.
  • "The format matters as much (or more) than the content of what you write.

"It's no wonder so much business writing winds up dry, wordy, and dripping with nonsense. People are just continuing continuing the bad habits they picked up in school. It's not just academic writing, either. There are a lot of skills that are useful in academia that aren't worth much outside of it.

"Bottom line: The pool of great candidates is far bigger than just the people who completed college with a stellar GPA. Consider dropouts, people who had low GPAs, community-college students, and even those who just went to high school."


"If you are trying to decide among a few people to fill a position, hire the best writer. It doesn't matter if that person is a marketer, salesperson, designer, programmer, or whatever; their writing skills will pay off.

"That's because being a good writer is about more than writing. Clear writing is a sign of clear thinking. Great writers know how to communicate. They make things easy to understand. They can put themselves in someone else's shoes. They know what to omit. And those are qualities you want in any candidate.


"You can't hide anymore. These days, someone else will call you on it if you don't do it yourself. They'll post about it online and everyone will know. There are no more secrets."


"Instant cultures are artificial cultures. They're big bangs made of mission statements, declarations, and rules. They are obvious, ugly, and plastic. Artificial culture is paint. Real culture is patina.

"You don't create a culture. It happens. This is why new companies don't have a culture. Culture is the byproduct of consistent behavior. If you encourage people to share, then sharing will be built into your culture. If you reward trust, then trust will be built in. If you treat customers right, then treating customers right becomes your culture.

Culture isn't a foosball table or trust falls. It isn't policy. It isn't the Christmas party or the company picnic. Those are objects and events, not culture. And it's not a slogan, either. Culture is action, not words."


"A lot of companies post help-wanted ads seeking "rock stars" or "ninjas." Lame. Unless your workplace is filled with groupies and throwing stars, these words have nothing to do with your business.

"Instead of thinking about how you can land a roomful of rock stars, think about the room instead. We're all capable of bad, average, and great work. The environment has a lot more to do with great work than most people realize.

"That's not to say we're all created equal and you'll unlock star power in anyone with a rock star environment. But there's a ton of untapped potential trapped under lame policies, poor direction, and stifling bureaucracies. Cut the crap and you'll find that people are waiting to do great work. They just need to be given the chance.

"This isn't about casual Fridays or bring-your-dog-to-work day. (If those are such good things, they why aren't you doing them every day of the week?)

"Rockstar environments develop out of trust, autonomy, and responsibility. They're a result of giving people the privacy, workspace, and tools they deserve. Great environments show respect for the people who do the work and how they do it."


"Write to be read, don't write just to write. Whenever you write something, read it out loud. Does it sound the way it would if you were actually talking to someone? If not, how can you make it more conversational?"

Sunday, May 20, 2012

Bits from the beginning of Five Minds for the Future

Five minds for the future
Howard Gardner

This guy keeps going with his multiple intelligences thing. I haven't finished the book, but here are some selections I wanted to remember:


"In the long run, it is not possible for parts of the world to thrive while others remain desperately poor and deeply frustrated."


"In a world where we are all interlinked, intolerance or disrespect is no longer a viable option."


"'Education is inherently and inevitably an issue of human goals and human values.' I wish that this statement were mounted prominently above the desk of every policymaker. One cannot even begin to develop an educational system unless one has in mind the knowledge and skills that one values, and the kind of individuals one hopes will emerge at the end of the day. Strangely enough, however, many policymakers act as if the aims of education are self-evident; and as a consequence, when pressed, these policymakers often emerge as inarticulate, contradictory, or unbelievably prosaic. How often my eyes have glazed over as I have read vacuous proclamations about 'using the mind well' or 'closing the achievement gap' or 'helping individuals realize their potential' or 'appreciating our cultural heritage' or 'having the skills to compete.' Recently, in speaking to ministers of education, I've discovered a particularly Sisyphean goal: 'leading the world in international comparisons of test scores.' Obviously, on this criterion, only one country at a time can succeed. To state educational goals in this day and age is no easy undertaking; indeed, one purpose of this book is to posit several more gritty goals for the future.

"A first caveat: science can never constitute a sufficient education. Science can never tell you what to do in class or at work. Why? What you do as a teacher or manager has to be determined by your own value system - and neither science nor technology has a builtin value system. Consider the following example. Let's say that you accept the scientific claim that it is difficult to raise psychometric intelligence (IQ). From this claim one can draw two diametrically opposite conclusions: (1) don't bother to try; (2) devote all your efforts to trying. Possibly you will succeed, and perhaps far more easily than you had anticipated. Same scientific finding: opposite pedagogical conclusions."


"To the extent that personal service or personal touch continue to be valued, these disciplines will provide a good livelihood for those who have mastered them. But my focus here falls chiefly on the scholarly disciplines that one should acquire by the end of adolescence, and the one or more professional disciplines needed to be a productive worker in society."


"Most important, set up 'performances of understanding' and give students ample opportunities to perform their understandings under a variety of conditions. We customarily think of understanding as something that occurs within the mind or brain - and of course, in a literal sense, it does. Yet neither the student nor the teacher, neither the apprentice nor the master, can ascertain whether the understanding is genuine, let alone robust, unless the student is able to mobilize that putative understanding publicly to illuminate some hitherto unfamiliar example. Both teacher and students ought to strive to perform their current understandings; much of training should consist of formative exercises, with detailed feedback on where the performance is adequate, where it falls short, why it falls short, what can be done to fine-tune the performance.

"Why talk about performances of understanding? So long as we examine individuals only on problems to which they have already been exposed, we simply cannot ascertain whether they have truly understood. They might have understood, but it is just as likely that they are simply relying on a good memory. The only reliable way to determine whether understanding has truly been achieved is to pose a new question or puzzle - one on which individuals could not have been coached - and to see how they fare."


"When critics deride business schools as being too academic, they usually mean that the ultimate uses of the purveyed knowledge are not evident; students are not forced to flex their text or lecture- or discussion-obtained knowledge. Here, in brief, is why most standardized measures of learning are of little use; they do not reveal whether the student can actually make use of the classroom material - the subject matter - once she steps outside the door. And here is why traditional training in the crafts requires a culminating masterpiece before the journeyman can rise to the level of master."


"As Plato remarked so many years ago, 'Through education we need to help students find pleasure in what they have to learn.'"

Friday, May 18, 2012

Quotes from Frederick W. Kantor

One of the most interesting things to come out of teaching a New York Academy of Sciences after-school math program at Ronald Edmonds middle school in Brooklyn was quite entirely unexpected.

As I was introduced to people on the first day of the program, some pride was shown by the administrator when she indicated a heavily bearded, wizened old man sitting next to a girl having trouble with fractions or some such thing. Doctor Kantor, she said, who had all those things published in the journals.

Who?

The very same Dr. Frederick W. Kantor of Columbia University, famous in some circles for originating the field of Information Mechanics. (The wiki page is now called Digital Physics; I don't think Dr. Kantor would approve.) The astonishingly low chance of running into an innovative physicist at a New York City public middle school, and then one whose work I find so interesting, was enough to make me think the universe was having a laugh with me.

I don't always follow everything Dr. Kantor says, but just yesterday after the last of this after-school program's sessions we chatted for a bit and it was really quite enjoyable. Here are two quotes, the second of which is him quoting Chien-Shiung Wu:

"A theory that tries to explain its assumptions is a tautology."
- Fred Kantor

"They forget that mass is an assumption."
Madame Wu to Fred Kantor

Dr. Kantor also spoke of some newer work of his, especially that relating information mechanics and the foundations of mathematics, which I really hope he publishes soon. I'm no expert, but I think information mechanics is a candidate for being a Really Good Idea that people have been neglecting.

Thursday, May 17, 2012

Brief thoughts on Simulacron-3

Simulacron-3
Daniel F. Galouye

I finally FINALLY got a chance to read Simulacron-3 after my paperbackswap.com request went through. It only took a few years!

This book is the basis for, among other things, the movie The Thirteenth Floor. It's arguably also the inspiration for The Matrix and a lot of related thinking. I'd be curious to know if there are proper antecedents to Simulacron-3, beyond abstract philosophers.

It's interesting reading a book from 1964. The basic idea of simulating a whole world is pretty interesting and modern-feeling, but the gender roles are archaic and computers are basically imagined as bigger versions of that era's, with physical components that you change like a lightbulb and programming basically by wire. And of course there are moving sidewalks everywhere, because that's what we do in the future.

Two quotes to pull out:

"You can hardly stuff people into a machine without starting to wonder about the basic nature of both machines and people." (p. 24, and repeated again later)

"For self-awareness is the only true measure of existence!" (p. 140)

Interesting that the characters care so little for the people in the simulation within a simulation, when they're so concerned about the people in the first simulation... But I am glad I finally encountered this book in person.


Monday, May 7, 2012

Reinvent standardized tests for openness

Identical tests kept secret until administration should be replaced by large open banks of test questions an appropriate sampling of which are faced by individual test takers. The only secrecy that should remain is in the particular combination of questions that the individual test taker faces when taking the exam.

It is always possible for test content secrecy to be compromised. There seem to be such problems now in California. It is claimed that breaches in security "could lead to invalidating test scores for entire schools or prevent the state from using certain tests." A system in which this can happen is not a robust system, and it is always possible for this to happen in a system that relies on test content secrecy. Perfect test content secrecy is not possible.

Limiting access to test materials before administering exams, especially when this means limiting it to companies motivated to produce tests at the lowest possible expense, does not effectively prevent  problems with the exams. And since there is only one version of the exams, problems affect every test taker. This has been the case with English and math tests recently in New York.

A far better solution to testing is to curate large banks of questions accessible to the public at large. This would allow many eyes to identify problems with questions, and give every student a fair chance to prepare for the kinds of questions that will appear on their exam.

The number of questions should be large enough that the probability of a test taker encountering a previously viewed question is fairly small. A test taker, especially one who prepares voraciously, may recognize a question during an examination, but this can also happen with secret content tests, and in any event it is the result of the test taker's learning.

If the banks of questions are sufficiently large and rich, this method may ameliorate the problems of "teaching to the test". It should become clearly more efficient to learn general principles and problem-solving skills, rather than simply reciting every existing question. Of course some practice with questions from the banks for practice is not necessarily bad, but by itself such an approach is unlikely to yield optimal results.

Since taking the test does not yield any new information about the test anyone else will face, it becomes possible for test takers to take and retake exams at any time with no new costs in exam development. This eliminates problems of exam-day sickness and the like.

Appropriate scoring of exams may be more difficult since test takers do not face the same questions. An intelligent and open system of evaluating question difficulty should be developed and marking should be aligned with the purpose of the exam. It may not be that the goal of the exam is to perfectly rank every test taker, and it may be easier to determine if a test taker has demonstrated competency and is ready to move on.

The modern world is advancing where transparency increases, and so should it be with testing.

Wednesday, March 14, 2012

More on educating for cognitive/behavioral outcomes (not just content knowledge/skills)


“Human history,” said H.G. Wells, is “a race between education and catastrophe.”
(as quoted in the Washington Post blog)

Education, I think we can agree if we take the time to phrase it, is about more than knowing things - it is also about the way in which we think, and the behaviors that we exhibit through our lives.

One curriculum that seems interesting is the Tools of the Mind program coming out of Denver, and now being implemented in DC. It builds on Vygotsky's ideas about mental tools and is thoroughly modern and so on, developing self-regulation in young children. I think it's great, but I can't help noticing that it seems similar in practice to Montessori methods. Montessori uses a concept called "normalization":

'Normalization arises from concentration and focus on activity which serves the child's developmental needs, and is characterized by the ability to concentrate as well as "spontaneous discipline, continuous and happy work, social sentiments of help and sympathy for others."'

Whatever you say the theory behind it is, I think it's great that educators are thinking about these kinds of learning goals. I see things reported of the type "study shows certain type of activity improves cognitive abilities of students" and I think "why aren't all the schools having their kids do that?" I've recently started messing around with Lumosity, which is trying to be a sort of cognitive training program for people of any age. It's pretty fun - and possibly really cool for the brains of the world.

I often think of better thinking as a goal of math education - it's not just for math, of course. Nearby math, people need good scientific thinking. I liked this article about making science education more scientific:
'We need a more scientifically literate populace to address the global challenges that humanity now faces and that only science can explain and possibly mitigate, such as global warming, as well as to make wise decisions, informed by scientific understanding, about issues such as genetic modification.
...
'The particular intervention we have tried addresses student beliefs by explicitly discussing, for each topic covered, why this topic is worth learning, how it operates in the real world, why it makes sense, and how it connects to things the student already knows.
'No matter what happens in the relatively brief period students spend in the classroom, there is not enough time to develop the long-term memory structures required for subject mastery.  To ensure that the necessary extended effort is made, and that it is productive, requires carefully designed homework assignments, grading policies, and feedback.
'As a practical  matter,  in a university environment with large classes the most effective way for students to get the feedback that will make their study time more productive and develop their metacognitive skills is through peer collaboration.'


So there's some thinking about how to get people to really think scientifically. Less far afield than you'd think, here's what Ira Glass relates in explaining what makes a good story:

'The story has to have more in it than “here’s what they do.” They need to make up theories about the interviewees, Alex says, putting them in categories, comparing them with other things, attaching them to bigger ideas. They need to always be thinking “this is like this,” “this means that,” “this little thing is an example of this bigger thing.” Especially “this little thing is an example of this bigger thing.”'

That kind of cognitive stance in the world, analyzing experiences and events around us, understanding things more deeply and making connections, is a desirable outcome of a good education.

There is a free curriculum available online that tries to address some of these goals, called Connections: Investigating Reality. I don't know if it's perfect, but it is at least a bold attempt to do something really cool with education. I like this list they have, a sort of collection of design principles for the course:

  • The future will be more complicated than the present. Old solutions won’t solve new problems.
  • To solve problems, you need to make sense of the real world.
  • In the real world, everything connects. You’ll need to understand “systems.”
  • Because they’re the creators of all sciences and all arts, human societies are the most important systems you can study.
  • Making sense of systems requires organized thought. School subjects aren’t very good organizers.
  • Thinking about ways to organize thought improves how you do it.
  • For sense-making purposes, the real, everyday world is a better “textbook” than textbooks about it.
  • Everything you learn should be useful, right here, right now.
  • Writing makes you think. (Keep a journal.)
  • Dialog makes you think. (Work with others.)
  • We’re not going to tell you much. We’re just going to give you a series of things to do and let you teach yourself how to make more sense of reality - yourself, others, the world

Learn-by-doing needs to be taken seriously, at least as a component of education, if not as the only component. There's a neat short TED talk about "studio schools" in the UK. Of course, not all activity is educational. And often the kind of activity that people do on their own is not what they need to do to learn new things. A sort-of-relevant quote from a piece about parent involvement that increases academic achievement:
'The kind of parental involvement matters, as well. “For example,” the PISA study noted, “on average, the score point difference in reading that is associated with parental involvement is largest when parents read a book with their child, when they talk about things they have done during the day, and when they tell stories to their children.” The score point difference is smallest when parental involvement takes the form of simply playing with their children.'

The above gives some education goals that are difficult to reduce to standardized test items. I don't think they are going to be particularly powerful without content knowledge and more basic skills (like adding, typing, reading, etc.) but neither will they necessarily develop if they are neglected. Let us educate well.


Sunday, March 11, 2012

Free Will Resolved

The universe has helped me out considerably by having Sam Harris write this book called Free Will, which saves me time since now I don't have to write it but I can still direct people to a book that explains my views on the matter. Hurrah!

I don't think that it's perfectly written, but it isn't awful, and I think the average reader should be able to understand the meaning from it, which is something. Of course it's all correct. Perhaps my favorite quote:
"You are not controlling the storm, and you are not lost in it. You are the storm."

Summary of the whole argument: Doing what you want and therefore choose is not what we usually think of as free will, because you don't choose what you will want in the first place.

Let us go forth, living more freely, having thrown off the shackles of free will!

Tuesday, March 6, 2012

Excel 2007 PERCENTRANK is trash.


People and software do not always mean the same thing when they talk about percentiles, percent rank, and so on. Do not expect different software to give the same values. In particular, Excel uses a method that is probably not what you expect and does not correspond to methods implemented in scientific software.

For Excel 2007, in the case of getting a PERCENTRANK for a value that appears in the range, you will actually get (the number of items strictly less than the value) / (the total number of items minus one). This has the nice feature of (at least for distinct values) giving percent ranks that range from zero to one inclusive. It has the nasty feature of almost certainly not being what you thought it was going to be, and not being what you'll get from SAS, R, SPSS, SciPy, etc. (It is, however, mimicked fairly well in other spreadsheet software.)

It isn't immediately obvious how Excel works out the PERCENTRANK for values that don't appear in the range. Some sort of interpolation, certainly - but not one that was easy for me to guess quickly. I'd love to know what the heck it is.

And it isn't just that Excel is non-standard - it also appears to be buggy. Here's one bizarre example I came across of Excel 2007 at work, in which PERCENTRANK is not stable when values are multiplied (or divided) by 100, sometimes giving the same percent rank for different values, sometimes giving different percent rank for the same values. Check out the rows in bold. You should be able to replicate this in Excel 2007 if you like (with nine digits of precision requested from PERCENTRANK).


value PERCENTRANK value/100 PERCENTRANK
96.775 1 0.96775 1
93.6625 0.954545454 0.936625 0.954545454
93.3 0.909090909 0.933 0.909090909
93.0125 0.863636363 0.930125 0.863636363
92.7875 0.772727272 0.927875 0.818181818
92.7875 0.772727272 0.927875 0.772727272
92.475 0.727272727 0.92475 0.727272727
92.0625 0.681818181 0.920625 0.681818181
91.5 0.636363636 0.915 0.636363636
91.275 0.59090909 0.91275 0.59090909
91.0875 0.545454545 0.910875 0.545454545
90.9125 0.5 0.909125 0.5
90.9 0.454545454 0.909 0.454545454
90.8375 0.409090909 0.908375 0.409090909
90.2625 0.363636363 0.902625 0.363636363
89.425 0.318181818 0.89425 0.318181818
89.0625 0.272727272 0.890625 0.272727272
88.4375 0.227272727 0.884375 0.227272727
88.1 0.181818181 0.881 0.181818181
83.325 0.136363636 0.83325 0.136363636
82.3375 0.09090909 0.823375 0.09090909
78.15 0.045454545 0.7815 0.045454545
71.5125 0 0.715125 0


The moral of the story? DON'T USE EXCEL FOR ANYTHING, BUT ESPECIALLY NOT MATH.