Ramblings on Learning Management Systems

This is seriously just very quickly edited text from an email I sent a while ago:



Date: 12 Dec 11 3:46 PM

First, consider two classes of skills or activities. One is "basic" skills that can be taught explicitly and practiced repetitively. I'm thinking of things like multiplying two-digit numbers, and I believe this class of skills can be taught and practiced really well with technology like Delta, Khan academy, etc. The second class are collaborative and "higher-level" skills that are best learned by working in teams on projects that have clear connections to the real world. I'm thinking of things like analyzing an issue, making a design, that kind of thing.

Now I grant that probably skills really lie on some continuous spectrum, but I think enough of the "basic" skills can be lumped together that the conceptual distinction can be useful.

To illustrate with math, problems with one correct answer are generally on the "basic" side and problems that admit of many solutions, like "design a system that uses data to evaluate NYC schools" or "engineer a device that solves this problem" are the other side. I feel like I want better names for these two categories. I propose "derivative" skills and "creative" skills. Derivative skills are skills you just learn from others, based on their work. Good. Also includes therefore knowledge of facts like "Mars is the next planet out in the solar system".

I think there is almost but not quite an analogy to pure and applied math. It falls apart because pure math of course has a creative leading edge, and applied math is about creatively using math to solve problems.

I think perhaps the pervasive corruption of debate on education is that some people seem to think that these two kinds of skill and education can succeed in isolation. And then you get kids with basic skills who can take a test but can't do anything creatively or solve problems in the real world, and you get kids who are creative but everything they create is complete crap because they don't have any supporting skills or knowledge.

I think that people who focus on derivative learning don't care much about the creative side because their kids do well on standardized tests. People focused on the creative side take one of two positions: they say either that basic skills are not important and shouldn't be tested, or that their kids learn them but somehow learn them differently through doing "rich" tasks.

And that last is my big gripe with some applications of constructivism, wherein educators set their students some rich task and expect it to take care of teaching the fundamental skills. I compare this to handing kids a chess board and expecting them to get good at chess just by messing with it. I'll have more to say about chess but first I'm curious if this all seems self-evident to you or if it seems like I'm missing something important. My claim here is that there are broadly two classes of important proficiencies, and that they might be best taught in two different ways.

**

Okay now the derivative skills form a foundation for further work, and I think it's fair to say that every student should have them. You should not have a few kids who know how to multiply and a few that always go to them to ask when they need something multiplied. (Never mind that this is effectively what a lot of managers do; I think we'd have a lot better managers if those folks had some skills too.) But this lack of even distribution of skills can easily happen with group work, as I'm sure you've seen. If somebody can already do it, I don't need to learn it. So I think that for many or all derivative skills, students should definitely demonstrate proficiency independently, and this will mean independent study and practice.

You can see that I'm getting toward learning management systems. What they already have is that they scale-ably (how is that spelled?) deliver individual interactive practice. They make attempts, to varying degrees, at delivering individual instruction (explanatory videos, etc.).

I think the as-yet unrealized great potential of technological solutions is to also deliver scale-able differentiation. They're the only possible solution I know of. One teacher can't work one-on-one with thirty kids at the same time. You're already doing much better than average because you can assign different kinds of problems on Delta to different students, but that still doesn't scale, because it is always more work for you to figure out what to assign and do so for every student individually.

What I envision is a system that incorporates some kind of machine learning with an internal representation of every student's current knowledge and skill set and customizes on the fly the experience that individual students have to optimize their progress toward educational targets.

What this means is: you don't tell students what exercises to do, and you don't tell the software what exercises to have the students do, you just sit the student down and tell the computer that the student needs to learn algebra, or whatever.

What would this be like? Well, it would be a little like Computer Adaptive Testing. If the student does something wrong, the computer makes an inference about what they know. If they do it right, the computer would make a different inference. So as an example, if you sit the student down to learn algebra and they already know it all, the computer will know that and tell you that the student is done. Bam. On to the next thing. But it isn't just Computer Adaptive Testing, it's Computer Adaptive Teaching, because based on what it infers about student knowledge the computer can offer different lessons, videos, activities, etc. to help move the student along.

This is a little bit like what School of One does; they have a huge library of computer (and classroom) lessons and assessments - but they spend a huge amount of time just deciding what each student should do the next day. Computer systems should be able to work this out more efficiently. Also, I think School of One has the wrong idea about the role and value of teachers. School of One makes sure to have teachers delivering lessons in the usual way, by talking and drawing on a board, so as to not "take anything away" from the teachers. But teachers shouldn't have to focus on these derivative skills. They should be facilitating the fun and interesting projects and other learning experiences, helping students with higher level skills. What they shouldn't have to do is worry that they can't get their kids to write a good essay because they don't know how to spell or capitalize properly.

What else is this like? It's a little bit like playing rated games against a chess computer. If I play and lose, my rating goes down and the next game I play is against the computer at a lower difficulty setting. This adjusting of difficulty makes it more fun for me (and there's a comparison to that idea of "flow" in matching difficulty to skill level) and by practicing I can get better at chess. But I would get better at chess more quickly if based on my play the computer also offered suggestions or lessons on how to play better in general. And this is possible.

So I imagine that we have a collection of lessons and a collection of assessments, and probably all or many of these are very short, like one question or a two-minute "lecture". (I'm not entirely sure that lessons and assessments really belong in separate bins; it might be better to have them all be together as "activities" or something to that effect.) And there's an internal space of concepts or proficiencies. This could be set up explicitly like "able to multiply six by seven" or "knows what thesaurus means" or "can find Atlanta on a map", or it could be completely generated by a machine learning algorithm and not necessarily correspond to our ideas of what the component skills are. The main point is that it will be more complicated than one value representing "skill in math" or something like that, the way your chess rating represents "skill in chess". But the system would learn based on user interactions that, for example, a student who can't do this also can't do that, or can do this but only after first doing that, and so on, so that a dynamic adaptive progression can be made for each student, and change as the student works.

I also think such a system would do well to incorporate ideas of carefully spaced repetition for maximizing retention in memory. Often a few repetitions at the right intervals are better than many repetitions in one bunch.

This is the outline of what I think could be a very good system for delivering real differentiated instruction in a way that scales across all students and can raise them all to very high levels of competency in basic skills. I think such an approach should be balanced with work on high-level rich tasks that build a whole suite of different skills, and I think it is in that realm that the importance of classroom teachers lies. In terms of time-share, I think it might be something like 30% individual work with technology, 70% collaborative project type work. (I think I'm not far off from what Khan has said on this.) (For one thing, the mental demands that this kind of computer work makes are very high. You can't just space out in the back of the room.)

I think that Delta and many similar projects are steps in the right direction toward enabling this kind of education, but I think there is a big missing piece and that piece is the kind of artificial intelligence approach that I outlined. So here's check-in number two: does it make sense, what I'm saying? Do you think it could be done? I think it would require a novel architecture, of course, but I think a lot of the components of Delta could fit into that architecture. What do you think would be the main hurdles of building something like this?

**

I think I sent you this link already: http://www.learningregistry.org/about because I think that project could also help form some of the content guts of a system like this. There is a lot of educational content out there, but solving the problem of getting that content to students at exactly the right time and evaluating whether they understand it is a difficult problem. It is a problem that I think could be solved by an approach like the one I'm describing. It looks like they have a name for systems of this type: a "learning management system".

added 2012.3.5: My best guess at what junyo is up to, with one of the founders of School of One, is that they're doing the big data machine learning data analysis buzzword xyz work toward some of the goals that I've mentioned for next-generation LMS.