This site describes the ILR scale upon which the DLPT, the DoD’s tool for measuring proficiency in a language, is based. It starts at 0 (no familiarity with the language) and goes up to 5 (functional equivalent to a well-educated native speaker), with half-steps between the steps, so there are 10 levels. The DLPT only tests up to level 3, “General Professional Proficiency,” described, in part, as being:
Able to understand the essentials of all speech in a standard dialect including technical discussions within a special field. Has effective understanding of face-to-face speech, delivered with normal clarity and speed in a standard dialect, on general topics and areas of special
interest; understands hypothesizing and supported opinions. Has broad enough vocabulary that rarely has to ask for paraphrasing or explanation. Can follow accurately the essentials of conversations between educated native speakers, reasonably clear telephone calls, radio broadcasts, news stories similar to wire service reports, oral reports, some oral technical reports and public addresses on non-technical subjects; can understand without difficulty all forms of standard speech concerning a special professional field.
So far, so good. But I’m not sure the DLPT is especially accurate in measuring these skills. For one thing, I scored a 2+ in listening and a 3 in reading in Korean, and I think at least half a point in each of those was due to good test-taking skills. (I am a monster on standardized tests.) For the most part, the test avoids the obviousness of the SAT, in which there are nearly always two answers that can be eliminated immediately. But the multiple-choice format means that a clever student who listens carefully for keywords and makes logical guesses to fill in the blanks may perform just as well as, or sometimes better than, a student who understands whole chunks of the passage. (Especially if he or she reads the question and possible answers in advance, in order to know what words to scan for. Again, test-taking strategies.) Moreover, there is a well-established core of intermediate and advanced vocabulary that regularly appears on the test; knowledge of that vocabulary, itself, can be enough to raise one’s scores from the 1+ level to the 3 level. (This happened to one of my military instructors, who lived in Korea for 13 years and was one of the best interceptors in the business, but who for years couldn’t pass the test, until he took a DLPT prep course at Yeon-Sei University.)
More critically, the test only measures skills up to level 3, and for good reason. Up to level 3, we are dealing, largely, with factual information. How much factual information can a person absorb from a given text (whether aural or written)? Even at level 3, where one is expected to follow “hypothesizing and supported opinions,” we are really dealing primarily with hypotheses based in fact, in the concrete — the subject matter may be more technical, requiring more advanced vocabulary, but the ideas are purely physical. “Scientists at Seoul University have discovered a gene that….” or “Diplomatic functions between Russia and Japan resumed this week after a standoff regarding….” or “The divorce rate in the United States was shown to be three times that of….” Actual opinion or attempts to describe human states, feelings, or spiritual conditions — beyond the level of “I like shopping at the mall because there are lots of stores” — make up, in my own estimation based on experience with the test, no more than 3-5% of the test; hardly significant, given the 15-pt. drop between perfect (65) and the minimum score for level 3 (50).
But it’s not clear, beyond this level, that there’s an easy correlation between knowledge of sophisticated vocabulary and the ability to follow sophisticated thought expressed in language. Consider, for example, the William Carlos Williams poem “This Is Just to Say”:
I have eaten
the plums
that were in
the iceboxand which
you were probably
saving
for breakfastForgive me
they were delicious
so sweet
and so cold
Apart from the slightly outdated term “icebox” (whose meaning, nonetheless, is readily available in its component words), this is a poem whose literal meaning can be easily understood by anyone with, say, a 1+ understanding of the English language. Yet it’s almost impossible to explain, in linguistic terms, why this poem is considered more sophisticated than Joyce Kilmer’s “Trees”:
I think that I shall never see
A poem lovely as a tree.A tree whose hungry mouth is prest
Against the earth’s sweet flowing breast;A tree that looks at God all day,
And lifts her leafy arms to pray;A tree that may in Summer wear
A nest of robins in her hair;Upon whose bosom snow has lain;
Who intimately lives with rain.Poems are made by fools like me,
But only God can make a tree.
“Trees” has, easily, the more difficult and obscure vocabulary of the two (archaic spellings and alternate words like “prest” and “lain,” uncommon words like “bosom,” species names like “robin”), as well as metaphorical content entirely absent in Williams’ poem. Yet “Trees” is almost universally regarded as an awful poem, sentimental and obvious, while Williams’ poem, though sometimes considered obvious or even pointless (see here or here for a variety of opinions), has the merit of being specific and evocative, at least to people in its intended cultural context.
And here is where, in more “advanced” or, certainly, more “literary” texts we start running into trouble. A level 4 speaker, for example, is expected to “discuss in detail concepts which are fundamentally different from those of the target culture and make those concepts clear and accessible to the native speaker,” while at level 5 he or she must be familiar with “pertinent cultural references.” Yet it is not at all clear how this can be made a function solely of linguistic skill. Even the most eloquent speaker of English may not necessarily understand that in America the icebox (later refrigerator) is a site of intrafamilial communication, a place where notes are typically left — may not even necessarily understand that notes are a necessary and frequent element of such communication. In a culture where note-writing in the home is not so common, it may not be immediately obvious from the title that this is such a note, as it is nowhere spelled out explicitly in the poem. But can this knowledge be legitimately classed as “linguistic”?
Moreover, it’s not clear that even two “highly articulate well-educated native speaker[s]” would receive this poem the same way. Leaving aside such amusing speculations as “this poem is actually about taking someone’s virginity,” the poem is ambiguous. The key question — is the writer truly sorry? — can’t really be answered. Does our understanding of the poem change depending on whether it is written for a loved spouse, an unloved spouse, an obnoxious roommate, one’s child? Obviously it does; the cruelty of “Forgive me they were delicious” is much greater if written to a child for whom they were a special treat than if written to a roommate who had drunk the last of one’s milk the night before. Lacking this information, what do we feel motivates such needling? The poem seems humorous, whimsical — but, the more we re-read it, the more a certain self-criticism appears. Williams the poet is poking fun at Williams the greedy plum-eater, at his self-centeredness and obliviousness to someone else’s feelings, despite his attempt to observe social convention by apologizing.
These are, obviously, more literary issues than linguistic. But how do we know we can separate the two? And at what point can we separate the prima facie meaning of a text from its, let’s say, significance? Perhaps this is not considered a problem in many government positions requiring language skill — after all, intelligence-gathering and analysis deal with facts, do they not? How many missiles, and made by whom, and of what vintage?
But they also deal with implication, particularly when dealing with an ideological enemy — and what other kind, this late in the game, do we have left?
The scale attempts to gloss over this problem by calling a level 5 linguist “functionally equivalent to… a highly articulate well-educated native speaker,” but this is simply a wordy way of saying, “He gets it,” without really defining what it is he gets.
The Handsome Camel 