Circular reasoning and the debasement of science

Ranking journals is a popular pastime among academics. Each of us has a favorite ranking, largely chosen by the results fitting with our favorite publication outlets. There are more debates over the methodology of journal rankings than of ranking business schools. There may be no universal agreement on the right method but there certainly is a wrong one.

Continue Reading

Kristie Engemann and Howard Wall have published a new ranking of economics journals. Their method consists of "a simple rule that considers citations only from a short list of top general-interest journals in economics." In short, they arbitrarily select the "top" journals, count the number of citations from these to other journals, add an adjustment here and there for effect, and presto! We determine the top journals by counting citations from top journals. Seems a bit circular.

If you walk into a random high school and want to know who the popular kids are, the Engemann and Wall method would have you identify them by seeing with whom the popular kids choose to hang out. The procedure might produce slightly different results if you started with the debate team than if you started with the cheerleading squad.^† It might not be a surprise, then, the top five journals in their results are included in the list of top journals by assumption. I don't disagree with the list, intuitively, but science should perhaps take a more objective path.

A more objective path does indeed exist. A commonly-used recursive algorithm initially assigns all journals an equal value. Each iteration of the algorithm assigns value from one journal to another based on citations. The iterative procedure, by the way, is at the heart of Google search results (replace "citations" with "links"). From the Google founders' monumental paper:

PageRank or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the principal eigenvector of the normalized link matrix of the web.

The authors of the new ranking poo-poo this mathy stuff:

[The iterative] procedure is largely a black box: It is not possible to see how sensitive the weights (and therefore the rankings) are to a variety of factors. The obvious objection to our rule is its blatant subjectivity. Our counter to this objection is to point out that the [iterative] procedure, despite its sheen of objectivity, contains technical features that make it implicitly subjective.

Ummm... Sensitivity analysis even has its own Wikipedia page.

If Engemann and Wall were to start their own search engine, the Google formula would presumably be replaced with "pages with links from pages we like."

---

^† This is not to say that cheerleaders don't often overlap with the debate team. But, seriously, they don't.

Hat tip, Mankiw

You're a geek if...

The setup: The Associated Press reported today that Minnesota is experiencing temperatures of forty degrees below zero!

The punchline: Is that Fahrenheit or Celsius?

If you laughed, you are (like me) a geek.

Whatever the scale, Minnesotans may be asking for a bit of that Global Warming to come their way.

On grading in a world less ordinal

We all have days where a small thing makes us unaccountably happy. One of those days, for me, was when we acquired a commercial-grade blender. It had but one speed: "on." After all, when I put something in the blender I just want it to come out, after a few whirling seconds, blended. This marked the end of unnecessary confusion with an old fourteen-speed blender, which always forced me to ponder whether the button labeled "whip" resulted in faster, slower, or roughly equivalent blending action to the one labeled "frappe." Why could it not simply be labeled "Speed 8"? After all, eighth gear on a ten speed bike is not called "expeditious" but simply 8th, comfortably nestled between 7th and 9th and respecting the natural order of integers.

How did Starbucks determine that "grande" is smaller than "venti" but bigger than "tall"? Even my four year old comprehends that "large" is bigger than "small," but understandably can't differentiate between subtle cross-cultural size differentials in translation. Why is Super High a higher frequency than Ultra High but lower than Extra High?

Yet, I have persevered and mastered the stand-ins for what should be facile ordinal comparisons. Until now.

I am confronted with another simplicity-defying reclassification.

Continue Reading

The lexicographically meaningful grades that used to reflect degrees of student performance (A comes before B before C) are being replaced with HP, LP, PA, and SP. Even an X is now offered as an option for the instructor, perhaps as a placeholder for those of us who relish single-letter simplicity. After introspection and even a bit of extraspection, I fail to comprehend how a less transparent recoding of presumably ordered evaluations aids learning, efficiency, or what's left of my hair.

My son's preschool teacher grades daily behavior on a scale of zero to four stars. Thinking that was perhaps too antiquated a numerical system, I relayed the teacher's notes to my son in a more contemporary fashion. "Grande effort following directions; cleaned up toys at frappe speed. Art project, regrettably, received only an HP." A quizzical look. "How many stars is that, daddy?"

He just doesn't get it. Neither do I.

Feel free to tell me what you think of my rant. Please rate this post on a scale of Elm to Pine.

UPDATE:I am not the only geek who values the ordinal aspect of grades. The visual brilliance that is xkcd today had this:

An unintended consequence of the new grading structure: this poor little stick figure no longer will find this bit of beauty in his life.

Where I make a pun about shaving

If William of Occam was alive today, I'm pretty sure he'd use something like this:

and not any of these:

See the rest

And before you go telling me that he would abandon all "shaving systems" (the industry has banned the term "razor") for a straight razor, recall that Occam preaches simplicity, not stupidity.

This post cannot be proven true

Randall Munroe has a solution to Gödel's incompleteness theorem, and to self-referential puzzlement in general.

First, John von Neumann's profound take on Gödel's result:

It was a very serious conceptual crisis, dealing with rigor and the proper way to carry out a correct mathematical proof. In view of the earlier notions of the absolute rigor of mathematics, it is surprising that such a thing could have happened, and even more surprising that it could have happened in these latter days when miracles are not supposed to take place. Yet it did happen.

And Randall's slightly more pithy (but no less profound) version:

See the rest

Wal-Mart’s «yesterday low prices»

I'm no marketing whiz, but this does not seem to be a sensible discount policy.

Why buy one when you can have two at trice the price?

Read on for more Wal-Mart "discounts"

These next two suggest that "saving" need not be financial. You'll pay just as much, but you'll feel better about it if we add something yellow to the sign. Everybody loves yellow.

And now we're just getting silly. Is "Rollforward" a word?

Hat tip Consumerist, Lee Aase, and random encounters

UPDATE (2 October 2007): Apparently, employee morale at Wal-Mart is low. Surprised?

Schrödinger’s cat outruns the police

I always suspected that my inchoate quest to understand quantum physics would pay dividends. When the police officer testifies at my next speeding ticket trial, I would repeatedly demand if he is sure, precisely, of where I was at the moment the offense occurred. When he finally convinces me, the judge, and all present, that he recalls my position with certainty, I'd have my Matlock moment. "Aha!" I would exclaim, explaining that he cannot, therefore, possibly know my speed, handing the Bailiff a well-worn copy of Heisenberg's article.

After reading Brian Green's Fabric of the Cosmos, I've discovered that my premise was correct, but not likely at the piddling speeds my car can attain.

Game theory foiled again

See the rest