This is the old blog

After about 400 posts here, I have decided to move to a different blogging platform. So I won't be posting here any more. I'll be posting on my new blog! Onward - to the future!

Fix R Tcl/Tk dependency problem on Mac OS X by installing working Tcl/Tk

For some reason, if you just install R on Mac OS X, you don't have a working Tcl/Tk by default. The problem manifests itself like this:


> tkplot(g)
Loading required package: tcltk
Loading Tcl/Tk interface ... Error : .onLoad failed in loadNamespace() for 'tcltk', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/Library/Frameworks/R.framework/Versions/2.15/Resources/library/tcltk/libs/x86_64/tcltk.so':
  dlopen(/Library/Frameworks/R.framework/Versions/2.15/Resources/library/tcltk/libs/x86_64/tcltk.so, 10): Library not loaded: /usr/local/lib/libtcl8.5.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/2.15/Resources/library/tcltk/libs/x86_64/tcltk.so
  Reason: image not found
Error in tkplot(g) : tcl/tk library not available

There are other functions that depend on Tcl/Tk as well. It's particularly annoying that you don't see the problem when you install or even load a library, but only when you try to use a function that depends on Tcl/Tk.

Fortunately, there's an easy solution. You need these two things:

1. X Windows for Mac OS X. You may already have this. If not, install it.
2. A Tcl/Tk installation that actually works. This will do the trick. Just install!

After you have these two installed, things should work. I didn't even have to restart R.

> tkplot(g)

Loading required package: tcltk

Loading Tcl/Tk interface ... done

Opinionated notes from the NYC Ed Tech Meetup

Context: A friend asked for highlights from the NYC Ed Tech meetup that happend earlier today, so I decided to make it into a blog post.

Tonight's meetup at Knewton was great! Here are highlights, in decreasing order of how much I like them:

• Dr. Bruce Wexler of C8 Sciences is a genius. His startup product, C8Kids, is a cognitive training web app along the lines of Lumosity, but way more sophisticated, more complete, and with better teacher interfaces for working with classes of students. It helps people make better decisions to help all kids, but especially kids with special needs, which is really cool. Not only is the project incredible, but Dr. Wexler was CONSISTENTLY the most articulate speaker, saying the right things at the right times and passing the mic when that was more appropriate. Lumosity should really just give their stacks of venture capital to C8 Sciences.
• Disclaimer: I have not actually used C8Kids, but it seems really great.
• Dr. Wexler also pointed me to BrainWare Safari, which has a kind of similar product but seems to be behind both C8Kids and Lumosity. Disclaimer again: I haven't tried it either.
• Dr. Wexler also mentioned ESCoNS2, the second meeting of the Entertainment Software and Cognitive Neurotherapeutics Society, which is happening in March in LA and has perhaps the most hippified web site I've seen in a while.
• Josh Baron from Marist (go raptors!) has done cool open source work using predictive modeling to identify students for interventions.
• Sakai is some sort of open source SIS/LMS thing. It seems like it could be good. I don't know. These sort of things have a tendency to get bloated. But: open source!
• Pentaho is some sort of open source business intelligence analytics thing. Maybe good?
• I had never even heard of PMML, the Predictive Model Markup Language. It is super cool that such a thing exists! At least in spirit, I'm totally in favor.
• OAAI is the Open Academic Analytics Initiative, which is maybe what all this is about. It's apparently their model which is supposed to be released in PMML, but I sure don't see it. I didn't search that long, either.
• Apparently there's some open source project called "Student Success Plan" but their web site is down. Possibly cool?
• The NYC Schools Gap App Challenge was totally in the house!
• I learned about things from people around me:
• Adium is apparently a chat client that some people use.
• Notability seems to be a very good iPad app for taking notes stylus-style.
• Most of Knewton's code is Java, it seems. Gross.
• Some IBM talk. connectedEDU, desire2learn, looking like you're wearing a three-piece suit even when you're not, blah blah blah. Okay there was some good message in there somewhere about united disparate data systems so that user experience is positive and the end result is good for kids, but mostly I wasn't feeling the corporate vibe and I started writing snarky tweets:
• tweet: "Bad metaphors are the new oil."#EdTechNYC Oh - also data. I hear it's a growth industry. Also, perhaps, education is good for children?
• tweet: Learning at #EdTechNYC: if you say "next chart" every 20 seconds through your presentation, you sound like an IBMer from the 1980s.
• Also somebody else tweeted about problems with a big IBM ed tech project, which sounded about par for the course: CALPADS 'in danger of failure'

Okay! So that was it! Pretty good! Thanks for the space and drinks, Knewton, but next time please provide plates to carry snacks on - I can't stand right by the snack table all the time! Important issue!

Upload images to your imgur account from the command line

If you just want to anonymously do a command line upload to imgur, use Bart Nagel's fairly complete imgurbash.sh, available along with many other tools on imgur's apps page. To make it work on my Mac I installed GNU sed with brew install gnu-sed and changed sed to gsed in the script, and it works fine, but it doesn't connect to your imgur account.

Another approach is to pull out an imgur cookie from a browser session and then sort of just pretend to be that browser session from some programming language. Lance Pollard wrote up something about this for Ruby. It might work. To me, it's a pretty ugly way to go about it.

I had never really done anything with OAuth2, so I felt pretty victorious when I got this to work. Alternate title: "Five minutes to OAuth2 uploads on imgur". I presume that you already have an imgur account for the following.

Step One: Register an application. The application is really just you, but that's fine. Choose "Desktop" for "Application Type". You should quickly get to a success page that gives you a "Client ID" and "Client secret". These are just strings that identify you; use them in the all-caps spots in subsequent steps.

Step Two: In a browser, go to this URL:
https://api.imgur.com/oauth2/authorize?client_id=YOUR_CLIENT_ID&response_type=pin
This is where you log in, convincing imgur that you are really you. You should quickly get to another success page that gives you a new string, this one being a "PIN".

Step Three: At the command line, enter this:
curl -X POST -F "client_id=YOUR_CLIENT_ID" -F "client_secret=YOUR_CLIENT_SECRET" -F "grant_type=pin" -F "pin=YOUR_PIN" https://api.imgur.com/oauth2/token
This will get you a JSON response which will include "access_token", which is the useful bit. You might also be interested in "refresh_token", but I'm not going there in this post.

Step Final: At the command line, enter this:
curl -X POST -H "Authorization: Bearer YOUR_ACCESS_TOKEN" -F "image=@PATH_TO_YOUR_IMAGE_FILE" https://api.imgur.com/3/upload
If cURL is talking about not being able to access your file, mess around with un-escaping spaces and stuff like that. It'll work. And this time you'll get a nice JSON response which includes an "id" and a "link" to your newly uploaded file - and that file will also now be appearing on your imgur account images page! This is especially good if you have a pro account and want your images to stay up indefinitely.

That's it! You can probably automate this further if you want. If you're building a web app, there are other better ways to interface with OAuth2. You can check the whole imgur API documentation.

Highlights from "This Will Make You Smarter"

This Will Make You Smarter
organized by John Brockman

I previously read and enjoyed John Brockman's What We Believe But Cannot Prove, so when I saw this volume as I was in line at Barnes and Noble, I picked it up. This Will Make You Smarter is not as good. The question was this: "What scientific concept would improve everybody's cognitive toolkit?" It isn't a bad question. He was looking for people to give novel "shorthand abstractions" such as "market", "placebo", and "random sample" once were. Most of the contributors didn't seem to understand the question. Many entries were not interesting or novel in the least. There was a lot of hemming and hawing over how un-scientific the general public is. I thought the last quarter of the book was by far the best. Here are a few of my favorite topics.

Kakonomics (Gloria Origgi) "the weird preference for low-quality payoffs"
This gives a word to that phenomenon wherein people claim to require and provide high-quality work, but everybody tacitly agrees to accept and produce low-quality work because it's easier. To be found, at least in pockets, in education, government work, and perhaps any sufficiently large organization.

Kafabe (Eric Weinstein) "an altered reality of layered falsehoods, in which nothing can be assumed to be as it appears"
He doesn't provide a better definition, I think, but the example that makes it clear is professional wrestling, in which everybody acts as if there's real competition but in fact the whole thing is planned. Maybe kayfabe is a specific case of Robert Trivers' entry in WWBBCP ("deceit and self-deception play a big role in human problems"), and then perhaps kakonomics is a specific case of kayfabe. Oh what a tangled web we weave?

Aether (Richard Thaler) (suggested as a pejorative for black box terms in theories)
This provides a word for use in criticizing theories that don't actually explain anything. Some other entries refer to the danger, well described years ago by Feynman, of thinking that naming a thing is the same as understanding the thing, and this is a little bit similar, but not exactly. I particularly like that he criticizes economics, saying that (among other things) time-varying risk aversion is an aether.

The Einstellung effect (Evgeny Morozov) "trying to solve a problem by pursuing solutions that have worked for us in the past, instead of evaluating and addressing the new problem on its own terms"
I didn't know there was a name for this. I do this kind of thing all the time. For example, if I think of doing a new web project, I immediately jump to Google App Engine, because I've used it before - even though it seems to not have much of a future, has a raft of clear drawbacks, and in any event is certainly not guaranteed to be the best tool for the job. But once you build one tool, you want to use it again rather than building a new one.

QED moments (Bart Kosko) "know[ing] what proof feels like"
This isn't really new either, but I like his angle on teaching the epistemological nature of mathematical proof as compared to other ways of knowing and how this sort of reasoning fits into the human experience.

The Veeck Effect (Gregory Cochran) "adjust[ing] the standards of evidence in order to favor a preferred outcome"
Yet another good name for something you already sort of know about.

Everyday apophenia (David Pizarro) the tendency to "err in the direction of perceiving patterns where none actually exist"
As in superstitions like lucky socks, etc.

Now that I go through my notes a bit, maybe it's me who didn't understand the question. The entries I found most interesting were the ones that gave names to phenomena I had some familiarity with already, not entirely new phenomena. One of the goals of the book was to create more cognitive shorthand, after all. Note also that these terms are useful in this - as, indeed, there is utility in having a term with which to refer to a specific type of bird. And perhaps it is too much to ask for such a book to provide entirely new concepts. Certainly really important new ideas would find expression in other outlets before being published in a compendium of this type. So there is some value in this tome, although I do maintain that it is not nearly as much fun as What We Believe But Cannot Prove. Still a fun series; I think I may check out What Have You Changed Your Mind About? next.

Art is Art: Two Galleries

I went to see Edward Tufte's gallery, ET Modern, at 547 West 20th Street. He's a sort of demigod of data visualization. Now he does art such as the above, which announces that "ART IS ART / AND EVERYTHING ELSE / IS EVERYTHING ELSE".

To get to ET Modern, I walked by the Jack Shainman Gallery at 513 West 20th Street. It looked interesting. I went in. It was showing a collection of work by Hank Willis Thomas called "What goes without saying".

Those are hand-painted blow-ups of old button designs. I explored these and other works, and then I went on to ET Modern to see Tufte's aluminum Feynman diagrams.

Tufte has made quite a lot of these aluminum Feynman diagrams. Some of them are hung like this, which brought to mind the most memorable thing I think I've ever overheard at a modern art museum:

Now this, this would go well with my sofa.

I do like Feynman diagrams quite a lot, to be sure. After I had gone to the two galleries once, I went back to "What goes without saying" to check it out again.

The second time at this gallery, I happened on a group that was being led by the artist himself, Hank Willis Thomas. It was a group of students whose professor knew the artist. I joined the group and felt welcome enough. Mr. Thomas talked about his work, something about his intent, his process, what he thought was good and interesting about various things - it was very nice. His work is not necessarily suitable for hanging over couches at country clubs. His work is communicative. It is emotional. It is meaningful. I am afraid my photo does not capture it very well. His work bears close examination. It bears consideration and reflection. At the very least, it has something to say.

I went to ET Modern once more to get a last picture. As I was leaving I asked the man who sat at a desk by the door, selling Tufte's books and things from the kind of ersatz gift shop there, why there was a big red "emergency stop" button by the exit. His explanation was a bit awkward, but he did demonstrate to me that it was completely non-functional, a kind of cartoon joke on the wall. What he said was, "It stops customers who don't buy things here."

I did think some about Tufte's work. "ART IS ART /  AND EVERYTHING ELSE / IS EVERYTHING ELSE" - I was more than a little tempted to declare that Tufte's work falls clearly on one side of that division, and that Thomas' work falls on the other. But there is still a lot to like in Tufte's work, his books, even his art.

"IT'S MORE COMPLICATED THAN THAT"

Success in Natural Language Processing is Human-Level Intelligence

There's a lot of talk about Natural Language Processing, NLP, using computers to deal with lots of text. The current state of the art is like cooking with only a colander and whatever ingredients fall out of the tree in your back yard. People are entirely too excited about a collection of weak-sauce results.

• Sentiment analysis is a joke on the unpopular kids (brands) desperate to know if people like them. "How many times did they say the words Nike and Love in the same sentence? Huh? How many?" AKA let's-reduce-all-human-discourse-to-one-linear-scale.
• More general word frequency analysis can be fun, just like the index of an arbitrarily long book. And you hit problems with grammatical changes right away, so you start using some clever stemming approach, and that either makes things better or worse, and the machine is sure as heck not going to know which it is.
• Co-occurrence? That's the best you got?
• Okay Google's machine translation is pretty cool, but Chinese Room is not real understanding or analysis.

Take a look at this public service ad on the NYC subway:

Here's what it says:

MTA

.info

What's next?

Poetry is back

in Motion.

Many of you felt parting was not such sweet sorrow.

So we're bringing poetry back in a very artful way.

Hopefully, you'll feel transported.

Improving, non-stop.

When I was in Korea and studying Korean, I tried to read and understand signs that I came across. If nothing else, I should be able to read the signs in the subway, right? Even if you speak English as a first language, if you don't regularly ride the subway in New York, you may not know what this sign is really saying. If you're clever, you can sort of guess, but probably not perfectly.

MTA is the Metropolitan Transportation Authority. This isn't stated in the ad, but humans can probably guess that this public-service-announcement-looking sign on the subway is probably from the subway people.

"What's next?" is a rhetorical question, and in fact a sort of MTA advertising series as they tell us what updates and changes are happening now or in the near future.

"Poetry is back in Motion" refers to the MTA "Poetry in Motion" series, a separate initiative that puts short poems in subway ad slots. Apparently they stopped doing this for a while, and now they're going to be back. Or maybe they've just failed to sell all the ad slots. Who knows.

Next our friends the MTA allude to Juliet's parting words to Romeo from her balcony. Apparently by the end of the play both Poetry in Motion and all subway riders will be dead. But seriously, this allusion just doesn't make sense. Does no one understand the feeling Juliet was conveying? Honestly?

The "feel transported" is actually kind of a fun double-meaning pun. The "non-stop" is less fun but would probably be better if it wasn't following a bunch of other junk just like it.

I suppose you could say that the MTA has really done a noble job in making a dull message a little more fun. No doubt. The point is that really understanding even this fairly simple message is not so easy. What if your NLP doesn't have the NYC-subway-rider plug-in? Or the dual-meaning-pun plug-in? The Shakespeare plug-in? I suspect that until machine text analysis is done by an embodied learning computer with human-equivalent intelligence, we will be limited to the frankly unimpressive kinds of tools that we have so far. To make my suggestion even less helpful, I suspect that as soon as such technologies exist, they will have the same drawbacks that humans do. Perhaps computer users will all have to be managers. Will I have to give my computer the weekend off? Maybe I should have... ramble mode OFF