the ladybug and the lion

On Being A Research Scientist.

You are comfortable with feeling like you have no deep understanding of the problem you are studying. Indeed, when you do have a deep understanding, you have solved the problem and it is time to do something else. This makes the total time you spend in life reveling in your mastery of something quite brief. One of the main skills of research scientists of any type is knowing how to work comfortably and productively in a state of confusion. 

From an answer on Quora to the question: What's it like to have an understanding of very advanced mathematics?

Best Abstract Ever.

 

Full paper here. 

Academia.

Reality.

A Critique Of Behavioral Economics.

Gigerenzer prefers to look for actual decision processes. Take the simple act of catching a ball in flight. The spirit of neoclassical economics would say that people act "as if" swiftly calculating the parabolic arc of the ball. The spirit of behavioral economics would explain dropped catches with references to some systematic errors in the way we perform that calculation. But in fact, ball-catchers use a cognitive short-cut called the "gaze heuristic," running forward and back while keeping constant the angle of sight up to the ball as it descends. No amount of "nudging" towards faster differential calculus will help prevent dropped catches.

This is tough on behavioral economists, because in order to be taken seriously by other economists they have had to play the optimizing game. Switching to Gigerenzer's rules would mean the end of economics as we know it.

Yet the critique is sobering. If behavioral economists do not really understand why we do what we do, there are surely limits and dangers to the project of nudging us to do it better.

Full article from Slate.

While a good critique, this doesn't invalidate all ideas developed by behavioral economists. And the authors seem to have missed the developing branch of economics that deals exactly with the problem of how brains process decisions: neuroeconomics.

The Peer Review Process.

Papers are increasingly being taken apart in blogs, on Twitter and on other social media within hours rather than years, and in public, rather than at small conferences or in private conversation. In December, for example, many scientists blogged immediate criticisms of another widely publicized paper² — this one heralding bacteria that the authors claimed use arsenic rather than phosphorus in their DNA backbone...

...To bring some order to this chaos, it looks as though a new set of cultural norms will be needed, along with an online infrastructure to support them. The idea of open, online peer review is hardly new. Since Internet usage began to swell in the 1990s, enthusiasts have been arguing that online commenting could and should replace the traditional process of pre-publication peer review that journals carry out to decide whether a paper is worth publishing.

Full article from Nature. It's worth reading.

(via The Browser)

Graduate School Rankings.

The National Research Council, on the other hand, might have done better to toss the idea of rankings entirely, and just clustered the departments into natural groupings. The statistician Leland Wilkinson ran a quick and dirty clustering on the NRC data for math departments. He found that the departments broke up into five clusters: 10 elite departments, a big group of 59 upper-tier departments, 47 lower-tier departments, and two smaller clusters whose meaning, if any, isn't clear to me. This is much coarser information than a full ranking—but it has the advantage of not depending on politically contentious choices as to which criteria matter most.

Full article from Slate.

CalTech.

What this means is that at Caltech, there are no dumb jocks, dumb legacies, or dumb affirmative action students. It is clear from its published statistics that the non-academic criteria that preoccupy admissions committees at all other elite universities count for little at this beacon of pure meritocracy. Perhaps the most striking difference from all other elite universities -- including institutions like MIT and the University of Chicago which forgo athletic recruitment -- is Caltech's complete indifference to racial balancing. In a state and a region of the country with the largest Hispanic population, Caltech's entering freshmen class in 2008 was less than 6 percent Hispanic (13 out of 236). The unwillingness to lower standards for a larger black representation is even more striking -- less than 1 percent (2/236) of Caltech's 2008 entering freshmen were listed as "non-Hispanic black". This "underrepresentation" of blacks and Hispanics, of course, was more than made up for by the huge "overrepresentation" of Asians. Only 4 percent of the U.S. population, Asians made up a whopping 40 percent of the incoming freshmen class in 2008, a slightly larger proportion than the 39 percent figure for whites. Applicants to Caltech are clearly seen as representing only themselves and their own individual merit and achievement, not their race or their ethnic group. As a professor at Caltech who has taught there for many years explained to me in an email, "We try, like our competitors, very, very hard to find, recruit, and nurture underrepresented minorities but we won't bend our standards."

Full article.

(via MR)

Complexity.

Our problem is that the scientific desire to simplify has taken over, something that Einstein warned against when he paraphrased Occam: “Everything should be made as simple as possible, but not simpler.” Thinking of natural and economic systems as essentially stable and decomposable into parts is a good initial hypothesis, current observations and measurements do not support that hypothesis—hence our continual surprise. Just as we like the idea of constancy, we are stubborn to change. The 19th century American humorist Josh Billings, perhaps, put it best: “It ain’t what we don’t know that gives us trouble, it’s what we know that just ain’t so.”

So how do we proceed? There are a number of ways to approach this tactically, including new data-intensive techniques that model each system uniquely but look for common characteristics. However, a more strategic approach is to study these systems at their most generic level, to identify universal principles that are independent of the specific details that distinguish each system. This is the domain of complexity theory.

Full article by theoretical biologist George Sugihara. It's worth reading.

(via The Browser)

Faith In Science.

A recent New Yorker article discusses the "decline effect": the observed decline in statistical significance of many scientific studies as they get replicated.

But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants: Davis has a forthcoming analysis demonstrating that the efficacy of antidepressants has gone down as much as threefold in recent decades.

For many scientists, the effect is especially troubling because of what it exposes about the scientific process. If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved? Which results should we believe? Francis Bacon, the early-modern philosopher and pioneer of the scientific method, once declared that experiments were essential, because they allowed us to “put nature to the question.” But it appears that nature often gives us different answers...

...Such anomalies demonstrate the slipperiness of empiricism. Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions. (Such shortcomings aren’t surprising, at least for scientists.) And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn.) The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.

Full article from the New Yorker. It's worth reading.

Along similar lines, here is a book that I've heard good things about (I haven't read it yet) on issues with statistical tests.