Tuesday, January 22, 2008

Center Exams, 2008

It was the season, and another rite of passage for 543,385 would be college students was conducted on this last weekend. This was the weekend of the Heisei Year 20 Center Exam. This exam is offered all over the country at exactly the same time. There are exams in History, Geography, Civics, Math, Science, Japanese and English. There are other foreign language exams offered, but very few people take them. Depending on a student's college entrance requirements, they choose which tests they must take.

The English test has two parts, a written test and a listening test. The written test is an eighty-minute exam and the listening is thirty minutes. Both are multiple choice. As a test, I am still baffled by many things. First are the aims of the exam. If it is supposed to be a test of language learned in junior or senior high school, the current exam doesn't make sense. This year there were no clearly ambiguous questions as there have been in years past, but it certainly isn't communicative. The Ministry of Education states that their aims for junior and senior high English are to get students to a communicate level in the language. This exam tests nothing of the kind. It would have no direct connection to the English programs at any college, either. It is a self-referential instrument for the purpose of discriminating between students. Since there is no transparency in the system, there is no evidence that the test is reliable, that it accurately measures language ability, or that it measures or predicts academic success.

That said, the logistics of the exam are a marvel, a tribute to planning and lots of money. Take for example the English listening test alone. According to the Japanese press, 498,800 took the test simultaneously. Test takers come to the test site at the appointed time. Tardiness means you forfeit the opportunity to take the exam and must wait until the next year. Each student gets a test booklet, a mark sheet, a personal audio player, and an IC chip memory card. At the exact time all students begin the listening test, a magical experience from a proctor's-eye-view. They don their earphones and silently obey the recorded instructions. Pages are turned in unison, answers marked, and pauses taken. Then in a synchronous movement, all of the test takers remover their earphones, place them and their pencils on the desk close their test booklets, and face front. All is silence. In a technological miracle, of the individual audio players distributed for the test, only 288 malfunctioned. Last year 1,254 machines were faulty. Four times fewer glitches in one year.

The exams were a logistical and technological miracle, but a pedagogical nightmare.


John said...

I agree with your views regarding the nature of the exams and would like to add that much the same applies to TOEIC. I will be invigilating this disaster of an assessment all this weekend and have generally spent the four hours in the exam room and the two hours of meetings wondering why they persist with something that clearly say nothing about language ability. The only positive note about it is the way the whole thing is conducted with military precision.

Daniel said...

Thanks for the comment.

You say that you will be invigilating the TOEIC all this weekend. Are they giving it several times at your place?

In these meetings is there any discussion about why they are offering the TOEIC?

I agree with you about the fuzzy connection between success on the TOEIC and language ability. One positive thing that can be said about the test is the transparency. There is a mountain of data available on the test on the Internet, and it is more reliable than the Center exams.

I'm also not sure why they needed two hours of meetings before the event. If you are proctoring the TOEIC IP test, it's pretty much of a no-brainer to do. It all comes in the box.

John said...

Yes, TOEIC is administered several times a year. I think, really, that a lot of staff time goes on coaching students to pass the TOEIC test. If they fail, retaking it is always an option.

As for meetings... it seems in Asia it's necessary to have meetings, and meetings about meetings. One part of the meeting was spent on all the invigilators introducing themselves (30 individuals), even though of course you only work with two other people. Team-building seems to be the thing. Apparently, academic staff have to be the ones to unseal all the boxes, count the papers, and reseal them. This can't be a job for administrative staff. And no, we don't get on to the effectiveness of TOEIC. Maybe this is because the test is so widespread, thus accepted, in Japan.

Daniel said...

Thanks again for the comment and answering my questions. I have loads more that I'd really like to ask, but you may not be possible for you to answer on this forum. And they're mostly motivated by morbid curiosity anyway.

According to the employers that I have heard from, any TOEIC score is better than nothing, and a higher one, over 400, is good, so if your school is trying to help the students out by boosting their scores a little, that's great. I don't know what else you are using it for, but you mentioned failing the test. Since the test isn't a pass/fail deal, the school must be using it for something else, probably beyond the parameters of the test.

Hang in there and help the students.

Jonathan said...

Hi Daniel,

fascinating stuff.
Do you have any information on how the students are being prepared for the listening component of the test? Is there an official curriculum being taught?

Would be great to connect directly to discuss this if you are open to it.

Daniel said...

I'm not sure how students are being prepared for this test specifically, and will guess that most students get some kind of "listening" test preparation at school and lots more at cram schools. My guess is that it also varies from school to school, and from student to student. Lots of kids are going abroad these days, but that is not training specifically geared to the Center test.

It'd be great to talk to you about it more with you. If you put your address in a reply to this comment, I will not publish it, and write you. I'd rather not publish mine here, and will be sure not to publish your's here