Just asking

The problem with standarized testing is multifold.
Problem 1 is universality of the testing. Do you test all students, including special needs (special ed/english as a second language) students? If so, should those scores of the special needs students be counted against the schools scores?
If you force those students to take the test, and count their scores, then schools that are magnets for such children end up receiving a worse grade than they deserve. It could also encourage schools to claim that they do not have the proper facilities to educate such children, which means that such children get *no* integration into a “regular” classroom. (Currently in the US special education students are integrated for one or two subjects into “regular” classrooms.)
If you say that those children do not have to take the test, or that their scores will not be counted against the school average, then you may have a problem of schools pushing low achieving students into special needs classes, or not releasing ESL students into mainstream classrooms–all because they may bring down the average of the school.
Problem 2 is teaching to the test. If *everything* is dependent upon how well a school does on one test, then the tendency is for teachers to teach to the test, which means that you may lose out on things that are not directly related to the test: students do work that is similar to what they will be tested upon, instead of work that may be more useful in the long run. (i.e. worsheets with problems similar to the test questions, instead of projects that actually apply the material.)
As my knowledge is of the US educational system, I gave rather broad answers about standardized testing in general, since I don’t know the specifics of Canadian standardized testing.
I come from a family of teachers (one of whom was a Canadian great-grandmother, so I’m not *comepletely* off base), so I get to hear a lot of this, whether I want to or not.
Michelle
Random (but not really)

Well, I seem to have hit a nerve here. I have responses, which I’ll have to post later. I’m at work right now, where there exist certain quantitative criteria that must be met in order for me to continue receiving a paycheque.

One more issue with standard testing hit my Mother pretty hard. She taught grade 5 and 6 English as a Second Langauge. Her kids, regardless of their time in Canada, had to complete the same test and we expected to hit the same numbers as native english speakers.
I know that’s not a general problem, it’s more of a policy issue, but nevertheless it’s related.
Tyler (forgot my login)

Re: Teaching to the test.
To further complicate the issue, here are two issues being raised here in Washington regarding the WASL (Washington Assessment of Student Learning) test.
The first is that this test is being used as part of the federal No Child Left Behind program that Dubya is so fond of. It boils down to this: state and federal funding is dependent upon the WASL scores. It’s seen as a way to make the school accountable for the academic success of their students, which is a great theory but it comes at a cost, particularly for schools already hit by budget cuts. And, trust me, a school that cuts its arts program to make ends meet is not going to reinstate it the instant they’re back in the black.
Besides, don’t you want your kid to go to the school that’s ranked #1 in 4th Grade aptitude?
The second problem is that, ultimately, the numbers are meaningless. This past spring the state lowered the scores required to pass. This just blows my mind. If this were an engineering experiment, if this was a crash test, you would not change the test in order to ensure that the product passes muster. And yet…
I just watch dumbfounded.
Chris
Click Tracks

standardized testing good.
Imagine if we took the “all students are special in their own way” approach to evaluating driving tests or medical board exams, airline pilots or bridge engineers?
Testing shouldn’t be thought of as a one shot deal. It would be better to think of it as feedback as opposed to a final evaluation. If you score badly there are areas you need to go back and work on and then you take the test again later. The test just shows where you are now, not tomorrow after a little more practice. There is almost always a tomorrow when it comes to failing a test but less so when you take those skill out in the world.
Both Joey and I would probably be destitute panhandlers if we were tossed out of academia for failing more than one time and thinking it was the end of everything as opposed to feedback saying we needed to drink and write for newspapers a little less and attend classes a little more. At least he would have the accordion thing to fall back on. Me, I would be probably just yelling at the dirty sock puppet on my hand.
Failure is also a fact of life. Seeing that the sun still comes up after one is an important learning experience. Shielding people by giving them soft testing just prolongs the inevitable and makes it harder for them to cope when it does happen. It also increase the chances that it happens on something bigger in life that is harder to rebound from.
Also when you screw with the feedback a person becomes unable to make informed decisions about what they should be doing. There are people out there that should never pursue math or writing or in my case, singing. Giving people milktoast generic feedback about their abilities is the equivalent of giving them career beer goggles.
Joey is a friend, so when we were in that bar that fateful night and I saw the girl looking at him and his accordion in an interesting, but not interested way, I gave him good feedback, which in that case was none. And I encouraged everybody at the table to do likewise. When Wendy looked at him at his christmas party a while later, we gave everybody in the house good feedback, run! She’s about to go through him like a bleach blond cougar through the side of a junior hockey team bus.
If someone is bad at something, let them know. Then they can decide if they want get better at that thing or try something else instead.
Take do it yourself investors, beginner self taught motorcycle riders, or those people who think they can fly by jumping off heights. A little unbiased feedback would have saved them a lot of pain later.
Also a standardized test sets a known quantity. You know, kind of like a standardized side of the road to drive on, a standardized colour of traffic light to stop at, or a standardized test for the strength of metal holding your car together. For computer types how much do you enjoy all your different standards? Or for employers of computer programmers, how useful is it to have a known standard level of programming abilities?
island airport bad. I first held the idea that people against it were a bunch of whiners and that for the area to grow economically we would need to let it grow. Then one day I was riding my bicycle along queen’s quay and the wind was blowing in from the south. There was a turboprop plane spooling up its jets out on the island. Even from where I was the noise was higher than would be allowed in any industrial workplace without protection. I feel for any property owner along there, it would substantially change their quality of living. When these people bought their homes here it wasn’t with intent of having window rattling ambient noise.
The solution is fast public transit out to pearson from downtown a la most first world major cities. Also no major city in the world seems to be suffering from a lack of a downtown airport, ie: London, New York. Or even Vancouver, Montreal and Calgary, all of which have viable public transit servicing their airports. It would be interesting to see where all the flights come and go from the island airport anyway. It’s not like you can fly toronto island / jfk direct. I am guessing that they are mostly private planes and small charters.
just a sensitive earred, passed a few more courses than I failed kind of guy,
Eldon

To answer the neglected question …
The issue was never the bridge itself but the island airport expansion that would become possible once the bridge was completed, including the use of small jets. Increased airport traffic is incompatible with the plan to build more mixed use neighbourhoods and parks along the waterfront.
If you would like to see this in action, in SimCity, build a waterfront town and include a small communter airport on an island just offshore. Once your city is humming along, upgrade to a larger airport and watch the neighbouring property values plummet.
Lob a meteor at the airport to relieve the air pollution!

Can we get the bridge without the airport?

The imagery was only to convey the intensity of the feedback. honest.
I think you make a good example with the regatta math problems. You are right that a standard test can be biased culturally. It is just that I think any test can be biased. The current standard ones or locally set ones. So it becomes a lesser of two evils thing. At least with a standardized test it can be updated more easily to remove biases that are discovered by getting lots of teacher feedback and it would suffer less from local biases and more people can work at making one test more objective. Some current biases in word problems may also be just outdating occurring too. Such as the one about the buggy whip factory production.
I think that local bias is the greater evil as it is way less likely to be as objective as a independently set third party test would be. This is why there is independent third party car crash testing or airplane testing.
It also comes down to why you are measuring or testing to begin with. If it is to evaluate individual teachers, schools or districts to determine which need more support then it would be necessary to compare them to others and a test that compares without a local bias in it becomes necessary. If it is to evaluate a student’s ability in certain subjects to see if they are prepared for university again it is to compare them to a minimum compentency that the universities expect.
A single standardized test that tries to sum up a students ability in a single number obviously has its shortcomings. So with things like the SAT there are problems. However a standardized test for each subject would be useful. A student applying to science and one applying to fine art could then both have the same total score if they added up their subject specific standard scores but then each would have a set of subject scores relevant to their abilities and interests. I believe the British system of A levels and O levels is based on standard exams by subject. Can any readers confirms this for me?
I also think people who think a single standard test score labels someone for life over a locally set test one when it comes to going on to unversity are kidding themselves.
First, I don’t think there are any universities that use a single factor such as a standard test score in determining admission. It is only one of a bunch that they use including essays, interviews, references, extra curriculars, volunteer work, etc…
Secondly, to overcome local test and grade creep biases that have been going of for years some universities handicap scores from the schools that students are applying from based on how students from that same school in the previous years did at that university. For example, if a 90% student from school 1 and a 70% student from school 2 both get a 75% in their first year the university will knock down the marks of future students from school 1 to reflect this.
Most of my points here are focused on the standard testing that comes at the end of high school and is used to go on to university. These are similiar to the tests that are used to go from university onto post grads, such as the GMAT or MCAT or LSAT etc. From the end of high school onward standardized testing is the most prevelent as it is when the greastest variety of students from the most different locations are trying to be evaluated and compared to each other and a common yardstick is necessary to do this. They may not all be applicable to the role and needs of testing in schools at lower levels where it is more about feedback for improvement.
Eldon

Meaning that you must produce at least 500 lines of code per day with no more than 1.2 bugs reported per day? Or meaning that you must, for example, make it easier to follow comment threads in Blogware posts?
The former would make a great standardized test. You count the lines of code, and then you count the bug reports filed against them, and you divide. The latter is so wishy-washy that any decent test author would laugh — can you show me 8.2 units of “easier”? Come back tomorrow with a real metric, one we can measure. Preferably with Scantron sheets.
And hey, I just noticed that this numbers-versus-results analogy even provides examples of the cultural bias The Redhead is talking about! Measuring strict lines of code means that a bad Python coder who writes 20 lines of sloppy Python (with a big data-loss bug) will show up as being more “productive” than a good Perl coder who turns out 5 lines of well-styled Perl (with one small formatting bug) to do the same task. Even better, the test proves both of them need to learn from a real programmer: that guy in the corner whose 150 lines of C contain a remote root exploit.

I’m sort of insulted, Joey, that you assume I don’t know that LOC is a meaningless metric. In fact, the phrase “5 lines of well-styled Perl” should be a dead giveaway that I’m not arguing from experience here.
I think I didn’t state my point well, so I’ll restate it: meaninglessness generalizes. LOC is a stupid measure of programmer ability because it doesn’t provide an indication of quality — it just spits out a number. And that number is useless. But this isn’t a problem with the LOC metric specifically, that scantron tests or provincial essay-writing tests with a standard marking guide magically avoid. The rot goes through the entire tree.
Scantron tests don’t measure student ability. They give a measure of circle-filling ability. (And they don’t provide good life experience, unless most of your life consists of picking from exactly four clearly different and easily-distinguished choices, two of which are obviously wrong. I don’t know about you, but I don’t spend most of my time looking at restaurant menus that offer a choice of Hamburger, Styrofoam, Sheep, or Poison.)
Essay questions, if they have a single marking guide that removes all subjectivity, test the ability of the student to guess what’s in the marking guide — Kevin Drum provided a critique by way of example a while back of how this can go wrong. And if they don’t have a standard marking guide, so that each person grading gets to use their judgement… well, we already have that. It’s called the system as it exists already, without standardized testing.
It’s true that math questions, with a single marking guide, come close to being regular enough for standardization… but even then, there’s slippage. Math-test evaluators could spend hours arguing over whether to give a higher score to student A who got the right answer with the wrong method or student B, who used the right method but produced the wrong answer. Lucky guess vs. close enough, triangle wins.
…
But there’s always a footnote, isn’t there? The caveat here is that most standardized-testing proponents talk about the tests as a performance review rather than a diagnostic tool. Fill in enough of the right bubbles and your school gets extra cash in the next state budget. Fill in too few and your school doesn’t get extra cash, and from what I’ve heard about them, your parents don’t get test results with sufficient explanation to improve your situation.
It’d be nice if standardized testing produced better students, but all it seems to produce is better test-takers. And you can take that to the zeppelin hangar.

I think that it’s important to separate the various issues being discussed here:
(1) Standardized testing.
(2) Interpretation of results.
(3) Use of (interpreted) results for evaluation of students’ performance.
(4) Use of (interpreted) results for evaluation of schools’ performance.
(5) Use of evaluation of schools’ performance as a criterion for funding decisions.
Often these all get wrapped up together. But they really are different topics, and linking them together is a policy (i.e., political) decision.
To address a couple of the points made above:
Yes, there are cultural biases in some tests. But is it necessarily the case that all standardized tests must have statistically significant cultural biases? If so, is this also true of other kinds of tests? (If it is true, then this is not an argument against standardized tests, but against tests in general. If it’s not, then what essential difference is there between standardized and non-standardized tests that allow the latter to avoid cultural biases?)
There will always be identifiable segments of the population that do worse on any test than other segments. It is also true that there will always be a leading cause of death. Discuss. 🙂
More seriously, let’s turn it around and suggest that this “identifiable segment” phenomenon could actually be a good thing: it can provide a basis for arguing that the IS in question is in fact disadvantaged and should receive more attention/public assistance/whatever. As with any measurement, proper interpretation is vital.
Joshua
jmadden@ics.uci.edu

Well, every one seems to be talking about standardized testing, I figured I should mentioned your second point.
The bridge and the island airport.
I moved out of toronto last year, so My information may be a bit late and mostly based on memory. The people I spent most of my time with where very much against the bridge, because of its effect on the waterfront of the area. Essentially, part of the increased size of the airport would be to put severe restrictions on the area’s boating community.
Currently any one sailing, rowing, kayaking, or any other kind of water based transportation much avoid a reasonably large area in front of the landing strip. (for VERY good reason) The area currently is large enough to be a minor inconvenience for moving around but certainly not a major problem. However if the airport is increased, then that area will be increased by 4 times. The larger planes, would need additional space in case they crashed… and large masts.. and airplanes don’t go together.
The increase in size would effectively close of the western waterway.
Aside from that issue, the general issue was annoyance at the increased noise and air pollution.
If you want to meet people who are REALLY against the airport.. talk to any of the yachting associations in the area.
Alex
Alex

If you’re not going to expand the airport, then there’s no reason to build a bridge.
I fly in to Toronto Island once in a while, so for purely selfish reasons I hope they keep the airport as it is. It’s a real pleasure to fly there. Of all the big-city airports that Air Canada goes to, it’s by far the best I’ve seen.

Heh. One should expect people to feel strongly about education, given the importance of it. Education is important, and providing a good education to as many people as possible is an important leveler in trying to equalize opportunities.
Good teachers are essential to any hope of improving education, and need to be both recruited, and encouraged to stay in the profession (not only through pay, but through supporting resources to make the job better; poor work conditions will often do more to get folks out of a job than pay alone).
While I’d like to see reductions in tenure (and/or remove some other specific provisions in the local teachers’ contracts, like bonus salary if they’ve done administration courses even for non-administrators), I think concurrent moves to make being a teacher more attractive as a career are good ideas.
In some ways, standardized testing is a more general version of what my Prof in Atlantic Canadian history referred to as ‘megaproject syndrome’…. the belief that ‘at last, this one idea will make everything better’. Big problems seldom have simple cureall fixes where one change suddenly reverses a downward trend years in the making, yet politicians persist in claiming they’ve found one.

“the pay for teachers is slightly above abysmal”…
An Ontario public school teacher with two university degrees and 10 years in the system earns more than $72,000 for eight months’ work, participates in an insanely generous pension plan, and practically has to commit arson or necrophilia to be fired. You can’t see it but I’m doing the “World’s Smallest Violin” gesture.

Just to give you a little history on the Ontario Teachers’ Pension Plan: the province used to manage it for the teachers with a fixed annual return (I think it was approx. <2% per year -- but let me check that). Any surplus above that fixed rate went straight into the general account for the province (ie. pension funds deducted from salary went straight to other gov't programs -- without notifying the membership).
Anyone who has any investments or an RRSP knows a monkey could achieve a better return than 2%. The teachers' union woke up and realized how the money was being redirected and asked if they could give managing the pension funds a try.
A couple of years later and a few wise investments decisions, the teachers' pension is doing quite well.
So the "lavish" pension is the direct result of wise, self-directed management. Why not let teachers take advantage of smart business decisions? We expect CEOs to walk home with hard-earned bonus packages, what's so bad about similar results from collective (aka. union) effort?
Sorry but I can't comment on the arson/necrophilia angle. I have heard of a teacher out west who was teaching anti-semitic material. They got rid of him fairly quickly (I hope).
Re: the 10yrs == 72K salary. How much do you expect to be making when you're 35 years old? Only 50k?
You know teachers receive no pay during the summertime, don't you? Last cheque at the end of June, first cheque near the start of October. My favourite was when the NDP government deferred the last June paycheque until late fall in order to save some $$ via cash management. Try that in any other industry and you would have riots.
Just some food for thought while you're playing your violin concerto. It's pretty easy to dump on teachers -- who hasn't had a bad math/history/english/music/gym teacher at least once in their lives -- but the full picture that's necessary for sound public policy should be better understood.

Wow. I’m 33 and I’m making a smidgen more than $50K for working twelve months a year, which puts me ahead of the vast majority of Canadian working people. You might want to get a clue, bud.

In terms of industry variations, Canada’s economy is still largely based on resources (the hewers of wood, drawers of water type stuff). But resource-based economies will always be squeezed as any commodity environment will. Added value is where the big money is (eg. refinement of commodities; knowledge sectors; innovative technologies).
How to break out of the resource commodity squeeze? Education.
Granted, certain infrastructures in Canada haven’t been revised or revisited in years (healthcare, education) and could use some reflection. Teachers are highly visible, but hardly the cause of society’s ills.

The problem with the bridge to the airport isn’t the bridge. It is the people behind it. The Toronto Port Authority is a federal government non-profit corporation, an unelected body filled by patronage appointments, their mandate is to manage the port (not that the port of Toronto is important, but we have Dennis Mills to thank for setting up this body) and to use any surplus to improve the port, which includes the fading island airport. However, every year they have been in existence they have lost money, and every year, their deficit grows by about 7 per cent. They do their best to hide their mis-management on the web, but since they need to get a budget top-up from the City, it’s in the record, if you dig through budget reports. Their annual budget is around $20 million, but essentially they are bankrupt, and always have been.
So where’s the money to “build” a bridge come from? Why the City, that’s where. Threaten to build a bridge nobody wants or needs, and then don’t build it, and pocket the claimed $24 million.
After all, the TPA are the ones who’ve soaked the City for $8 million for a ferry terminal, y’know the one they haven’t built yet, though the Rochester Ferry started a month ago.
So, if you run across somebody who wants the bridge built, I’m sure I can set them up with a really computer lease package, see I know this hairdresser, and his brother plays hockey …
Jim

I don’t feel sorry for anyone who gets a nice salary for 10 months of work and can’t manage their money.