On Tuesday, my Erdos number dropped from infinity to four. That’s right: after four years of grad school, I am now officially published!

The article, “A New Phylogenetic Diversity Measure Generlizing the Shannon Index and Its Application to Phyllostomid Bats,” by Ben Allen, Mark Kon, Yaneer Bar-Yam, can be found on the American Naturalist website or, more accessibly, on my professional site.

So what is it about? Glad you asked!

Protecting biodiversity has become a central theme of conservation work over the past few decades. There has been something of a shift in focus from saving particular iconic endangered species, to preserving, as much as possible, the wealth and variety of life on the planet.

However, while biodiversity may seem like an intuitive concept, there is some disgreement about what it means in a formal sense and, in particular, how one might measure it. Given two ecological communities, or the same ecological community at two points in time, is there a way we can say which community is more diverse, or whether diversity has increased or decreased?

Certainly, a good starting point is to focus on species. As the writers of the Biblical flood narrative were in some sense aware, species are the basic unit of ecological reproduction. Thus the number of species (what biologists call the “species richness”) is a good measure of the variety of life in a community.

But aren’t genes the real unit of heredity, and hence diversity? Is the number of species more important than the variety of genes among those species? Should a forest containing many very closely related tree species be deemed more diverse than another whose species, though fewer, have unique genetic characteristics that make them valuable?

And while we’re complicating matters, what about the number of organisms per species? Is a community that is dominated by one species (with numerous others in low proportion) less diverse than one containing an even mixture?

There is no obvious way to combine all this information into a single measure for use in monitoring and comparing ecological communities. Some previously proposed measures have undesirable properties; for example, they may increase, counterintuitively, when a rare species is eliminated.

In this paper we propose a new measure based on one of my favorite ideas in all of science: entropy. You may have heard of entropy from physics, where it measures the “disorderliness” of a physical system. But it is really a far more general concept, used also in mathematics, staticstics, and the theory of automated communication (information theory) in particular. At heart, entropy is a measure of unpredictability. The more entropy in a system, the less able you will be to accurately predict its future behavior.

The connection to diversity is not so much of a stretch: in a highly diverse community, you will be less able to predict what kinds of life you will come across next. Diversity creates unpredictability.

To be fair, we weren’t the first to propose a connection between diversity and entropy. This connection is already well-known to conservation biologists. But we showed a new and mathematically elegant way of extending the entropy concept to include both species-level and gene-level diversity. It remains to be seen whether biologists will take up use of our measure, but whatever happens I am happy to have contributed to the conversation.

on July 2, 2009 at 2:56 amJoshuaI understand entropy as the rate at which a system degrades energy from more to less useful forms. Is this a proper understanding of the idea?

If so, are you claiming that a system’s unpredictability is a function of its energy flow? That is, as more energy is degraded by the system, the system’s complexity and diversity increase while its predictability decreases.

A related question:

Is there a connection between increased entropy and increased symmetry/beauty. For example, it seems that the more entropy a flower has, the more symmetry and beauty it has. At the moment when it’s “falling apart” the most, it is also displaying the most beauty, no?

By extension, is predictability inversely correlated with symmetry/beauty?

on July 2, 2009 at 9:15 amrafefurst@Joshua, interesting thoughts.

As entropy increases, symmetry also increases. Beauty is a bit more complicated because some people feel symmetry is beautiful and others prefer asymmetry. Still others find symmetry ‘fearful’ :-)

I have gone out on a limb and claimed that asymmetry is the root of all value, and similarly it is the root of complexity (even though we still don’t quite know how best to define complexity itself).

Symmetry breaking is talked about as an important concept in physics for explaining the specific and seemingly arbitrary (but not random) configuration of matter and energy in the universe, given that the supposed “universal laws” seem to be all about symmetry. Can’t say I understand it at more than a superficial level.

Aesthetically, I am a fan of both symmetry and asymmetry…

on July 10, 2009 at 12:07 amplektix@Joshua- You ask two very good questions.

Yes, entropy is sometimes viewed as the rate of degradation of energy to less useful forms, but this definition is unsatisfying because it doesn’t specify what “useful” means. The forms of energy that are useful to us are precisely those that come in predictable, harnessable forms, like light or electricity. Heat is not so useful because it is literally the random motions of individual particles.

So yes, your statement “as more energy is degraded by the system, the system’s complexity and diversity increase while its predictability decreases” is correct, though “complexity” is a slippery word.

Your second question is worthy of a blog post of its own, but I will give a short answer that the relationship between entropy and beauty is not so simple. Compete randomness (think white noise on a TV screen) is not much more interesting than compete predictability (e.g. an unchanging blue screen). True beauty requires a sophisticated mixture of order and disorder.

on July 27, 2009 at 4:28 amJoshuaI’ve been thinking about entropy a lot over the last couple weeks, and I’ve come to the conclusion that for a long time I’ve been conflating two measures of entropy into one, and thereby confusing the entire concept. The two measures are as follows:

(1) Total entropy:

The total amount of disorder in the system (which is always increasing if the system is closed– 2nd law)

(2) Rate of entropy:

The rate at which order is being transformed into disorder. This is basically a derivative of (1) and time.

I’m thinking about these two measures in relation to the “flower” in my previous post. When the blossom is at its apex of beauty/symmetry/complexity (the moment just before it first begins to whither), its rate of entropy (2) is at its highest level. At that moment, the flower has the most “order” available to be transformed into “disorder,” and its rate of entropy (2) is highest. As it begins to whither, less and less order is available to be transformed into disorder, so its rate of entropy (2) decreases.

But all the while the system’s total entropy (1) is increasing, however high or low the rate of entropy (2).

So, with that said….

@rafefurst: You write: “As entropy increases, symmetry also increases.”

I’m wondering which entropy you’re talking about here. If you’re talking about total entropy (1), then in a sense you’re claiming that symmetry is always increasing (because total entropy is always increasing), which clearly isn’t the case, is it?

If you’re talking about the rate of entropy (2), then I would agree, but would flip around the statement:

As symmetry increases, the rate of entropy increases.

(think of the blossom above)

[Then again, maybe symmetry is always increasing… if you see the end state (after entropy is finished) as perfectly symmetrical, which it may very well be. For example, if the universe doesn’t contract and continues to expand and eventually all subatomic particles have broken down to their constituent parts (quarks and leptons and whatever) and all these particles are spread across the universe in perfect equidistance from each other…. at that moment entropy is complete, but we also have a completely symmetrical physical state, no?]

Re biodiversity: could you think of an ecosystem as a larger version of the flower above? As the ecosystem grows and diversifies by way of speciation, its symmetries become ever more complex and and its future ever more unpredictable. At some point, it reaches a kind of apex or peak where it has the most available order (complex symmetries) to transform into disorder. From that point on, it goes into decline as total entropy continues to increase while the rate of entropy slows down with each ratchet down in diversity and complexity.

on July 27, 2009 at 8:59 amrafefurstYes, this is the sense in which I meant higher entropy means higher symmetry.