Comments on: In Praise of Numeracy http://landshape.org/enm/lay-down-the-hockey-stick/ The Power of Numeracy Tue, 07 Feb 2012 17:57:00 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Levi Waldron http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5785 Levi Waldron Sat, 22 Mar 2008 23:16:11 +0000 http://landshape.org/enm/?p=46#comment-5785 Darn, not sure how to display R code in the comments... will try again with mediawiki markup... brownian Darn, not sure how to display R code in the comments… will try again with mediawiki markup…

brownian

]]> By: Levi Waldron http://landshape.org/enm/lay-down-the-hockey-stick/#comment-6138 Levi Waldron Sat, 22 Mar 2008 23:16:00 +0000 http://landshape.org/enm/?p=46#comment-6138 Darn, not sure how to display R code in the comments... will try again with mediawiki markup... brownian Darn, not sure how to display R code in the comments… will try again with mediawiki markup…

brownian

]]> By: Niche Modeling » Trends in Predictive Analytics 2006 http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5784 Niche Modeling » Trends in Predictive Analytics 2006 Thu, 13 Jul 2006 14:40:23 +0000 http://landshape.org/enm/?p=46#comment-5784 [...] Mike Levin at hittail.com has developed a unique site delivering predictive analytics for web sites. The concept is simple: place a small piece of code on your site that sends information back to the hittail server about your hits, and generate suggestions for web pages that will benefit you the most in terms of traffic. I have been trying it for a month at Niche Modeling, and I think this is going to be big — really big. Firstly: it replaces pouring through web analytics. All those graphs and urls never really helped achieve the purpose, which was to build traffic. Secondly: It has a theoretical basis in the longtail distribution of search terms. Thirdly: The AJAX interface at hittail.com is very slick and worth checking out! Basically the idea is this: some search terms result in your site on the first page of Google search results. Call this the head of the distribution. For some terms, your site is 5 or 6 pages down. By blogging on these topics you build your site and move those terms up in ranking to the front page. Over time, with more blogging and reranking by the search engines, your are on the front page of Google more often, thus building traffic. The suggestions generated by hittail.com maximize the benefit/cost ratio of your time. Topics you are already on the front page give less benefit than topics currently further down the search results. I see it hittailing a bit like investing in real estate. You already ‘own’ the terms that appear on the first page of a Google search. By blogging on the terms further down the search results you ‘acquire’ those terms, which over time builds the ‘assets’ of your site. The results I have been getting have been quite interesting. Niche Modeling is a specialist site on statistics and predictive modeling, largely about environmental topics like bird flu, climate change, geographic information systems (GIS) and some scientific fraud issues. If you try this Google search on avian influenza GIS data I get a post at the top of the results, above the post by Declan Butler who writes for Nature magazine, of which my post is largely derivative. Before my hits mainly came from click-through of links created on other blogs when I put up a particularly interesting post. This sort of traffic has a number of drawbacks. [...] [...] Mike Levin at hittail.com has developed a unique site delivering predictive analytics for web sites. The concept is simple: place a small piece of code on your site that sends information back to the hittail server about your hits, and generate suggestions for web pages that will benefit you the most in terms of traffic. I have been trying it for a month at Niche Modeling, and I think this is going to be big — really big. Firstly: it replaces pouring through web analytics. All those graphs and urls never really helped achieve the purpose, which was to build traffic. Secondly: It has a theoretical basis in the longtail distribution of search terms. Thirdly: The AJAX interface at hittail.com is very slick and worth checking out! Basically the idea is this: some search terms result in your site on the first page of Google search results. Call this the head of the distribution. For some terms, your site is 5 or 6 pages down. By blogging on these topics you build your site and move those terms up in ranking to the front page. Over time, with more blogging and reranking by the search engines, your are on the front page of Google more often, thus building traffic. The suggestions generated by hittail.com maximize the benefit/cost ratio of your time. Topics you are already on the front page give less benefit than topics currently further down the search results. I see it hittailing a bit like investing in real estate. You already ‘own’ the terms that appear on the first page of a Google search. By blogging on the terms further down the search results you ‘acquire’ those terms, which over time builds the ‘assets’ of your site. The results I have been getting have been quite interesting. Niche Modeling is a specialist site on statistics and predictive modeling, largely about environmental topics like bird flu, climate change, geographic information systems (GIS) and some scientific fraud issues. If you try this Google search on avian influenza GIS data I get a post at the top of the results, above the post by Declan Butler who writes for Nature magazine, of which my post is largely derivative. Before my hits mainly came from click-through of links created on other blogs when I put up a particularly interesting post. This sort of traffic has a number of drawbacks. [...]

]]> By: admin http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5783 admin Mon, 29 May 2006 02:40:38 +0000 http://landshape.org/enm/?p=46#comment-5783 Yeah well, longer than what? fatter than what? Thing is Chris Anderson's definition is a business with a large number of unique products, that emerges from low storage/distribution costs, compared with a business with few products. It doesn't necessarily meet the Wikipedia definition of having a preponderence of bulk (or value) residing in the tails. In the case of fat tails it is not enough to be infinitely long and approach zero. An exponential tail like the normal curve does that and is not problematic. Its more like distributions that decay slower than exponential that are the problem and are a feature of the strangly trendy series, and processes with costly extreme events that are more frequent than predicted by normal distributions. I think I will stop doing these teaser posts. Yeah well, longer than what? fatter than what? Thing is Chris Anderson’s definition is a business with a large number of unique products, that emerges from low storage/distribution costs, compared with a business with few products. It doesn’t necessarily meet the Wikipedia definition of having a preponderence of bulk (or value) residing in the tails.

In the case of fat tails it is not enough to be infinitely long and approach zero. An exponential tail like the normal curve does that and is not problematic. Its more like distributions that decay slower than exponential that are the problem and are a feature of the strangly trendy series, and processes with costly extreme events that are more frequent than predicted by normal distributions.

I think I will stop doing these teaser posts.

]]> By: admin http://landshape.org/enm/lay-down-the-hockey-stick/#comment-6137 admin Mon, 29 May 2006 02:40:00 +0000 http://landshape.org/enm/?p=46#comment-6137 Yeah well, longer than what? fatter than what? Thing is Chris Anderson's definition is a business with a large number of unique products, that emerges from low storage/distribution costs, compared with a business with few products. It doesn't necessarily meet the Wikipedia definition of having a preponderence of bulk (or value) residing in the tails. In the case of fat tails it is not enough to be infinitely long and approach zero. An exponential tail like the normal curve does that and is not problematic. Its more like distributions that decay slower than exponential that are the problem and are a feature of the strangly trendy series, and processes with costly extreme events that are more frequent than predicted by normal distributions. I think I will stop doing these teaser posts. Yeah well, longer than what? fatter than what? Thing is Chris Anderson’s definition is a business with a large number of unique products, that emerges from low storage/distribution costs, compared with a business with few products. It doesn’t necessarily meet the Wikipedia definition of having a preponderence of bulk (or value) residing in the tails.

In the case of fat tails it is not enough to be infinitely long and approach zero. An exponential tail like the normal curve does that and is not problematic. Its more like distributions that decay slower than exponential that are the problem and are a feature of the strangly trendy series, and processes with costly extreme events that are more frequent than predicted by normal distributions.

I think I will stop doing these teaser posts.

]]> By: TCO http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5782 TCO Sun, 28 May 2006 22:35:53 +0000 http://landshape.org/enm/?p=46#comment-5782 I think fat tail is the most intuitive, suggestive term. In either case the distribution (in some sense) is inifinitely long and approaches zero as an assmptote. Fat tail suggests that its not going down as fast as it "should". Fat tails killed LTCM. Well and not understanding that their portfolio parts were not really independant to certain risks. I think fat tail is the most intuitive, suggestive term. In either case the distribution (in some sense) is inifinitely long and approaches zero as an assmptote. Fat tail suggests that its not going down as fast as it “should”.

Fat tails killed LTCM. Well and not understanding that their portfolio parts were not really independant to certain risks.

]]> By: TCO http://landshape.org/enm/lay-down-the-hockey-stick/#comment-6136 TCO Sun, 28 May 2006 22:35:00 +0000 http://landshape.org/enm/?p=46#comment-6136 I think fat tail is the most intuitive, suggestive term. In either case the distribution (in some sense) is inifinitely long and approaches zero as an assmptote. Fat tail suggests that its not going down as fast as it "should". Fat tails killed LTCM. Well and not understanding that their portfolio parts were not really independant to certain risks. I think fat tail is the most intuitive, suggestive term. In either case the distribution (in some sense) is inifinitely long and approaches zero as an assmptote. Fat tail suggests that its not going down as fast as it “should”.

Fat tails killed LTCM. Well and not understanding that their portfolio parts were not really independant to certain risks.

]]> By: admin http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5781 admin Fri, 28 Apr 2006 16:46:10 +0000 http://landshape.org/enm/?p=46#comment-5781 Thanks for the comment Demetris. It really helps one gain an intution about the concepts to describe it that way. The behavior as a necessary result of entropy makes a lot of sense to me -- instead of seeing these behaviours as disembodied statistical distributions, that are an inevitable consequence of the 2nd law of thermodynamics. Or, from the point of view of information, a natural or complex system evolves in such a way as to minimize compressibility of its information. That is, a description of a natural system as a kind of machine with a distinct cycle or spatial scale (like a Markov process) should be the least accurate. Thanks for the comment Demetris. It really helps one gain an intution about the concepts to describe it that way. The behavior as a necessary result of entropy makes a lot of sense to me — instead of seeing these behaviours as disembodied statistical distributions, that are an inevitable consequence of the 2nd law of thermodynamics. Or, from the point of view of information, a natural or complex system evolves in such a way as to minimize compressibility of its information. That is, a description of a natural system as a kind of machine with a distinct cycle or spatial scale (like a Markov process) should be the least accurate.

]]> By: admin http://landshape.org/enm/lay-down-the-hockey-stick/#comment-6135 admin Fri, 28 Apr 2006 16:46:00 +0000 http://landshape.org/enm/?p=46#comment-6135 Thanks for the comment Demetris. It really helps one gain an intution about the concepts to describe it that way. The behavior as a necessary result of entropy makes a lot of sense to me -- instead of seeing these behaviours as disembodied statistical distributions, that are an inevitable consequence of the 2nd law of thermodynamics. Or, from the point of view of information, a natural or complex system evolves in such a way as to minimize compressibility of its information. That is, a description of a natural system as a kind of machine with a distinct cycle or spatial scale (like a Markov process) should be the least accurate. Thanks for the comment Demetris. It really helps one gain an intution about the concepts to describe it that way. The behavior as a necessary result of entropy makes a lot of sense to me — instead of seeing these behaviours as disembodied statistical distributions, that are an inevitable consequence of the 2nd law of thermodynamics. Or, from the point of view of information, a natural or complex system evolves in such a way as to minimize compressibility of its information. That is, a description of a natural system as a kind of machine with a distinct cycle or spatial scale (like a Markov process) should be the least accurate.

]]> By: Demetris Koutsoyiannis http://landshape.org/enm/lay-down-the-hockey-stick/#comment-5780 Demetris Koutsoyiannis Fri, 28 Apr 2006 10:49:41 +0000 http://landshape.org/enm/?p=46#comment-5780 Very interesting. I like the term "long tail" better than the more common in hydrology "heavy tail" and "fat tail" or the more technical "Pareto tail" and "power-law tail", so I will adopt it from now on. I think "long tail" is more natural and expressive than the other names. For example, one says that a cat has a long tail (not a heavy tail). In addition, typologically it is closer to "long-term persistence" and "long-range dependence" (that is, long-tail autocorrelation). Long tails in probability distribution and in autocorrelation function are two different things, both implying some scaling (power law) behaviour on the tails of these functions. In my couple of entropy papers in Hydrological Sciences Journal (2005) I used the names "state scaling" and "time scaling" for these two scaling types (the second could also be "space scaling" in a spatial process – if one substitutes space for time). Simultaneously, I tried to demonstrate in these that both long tails may be consequences of a single thing, the principle of maximum entropy, viz. maximum uncertainty. The combined effect of long tails in probability and long tails in autocorrelation, that is the dominance of maximum entropy, makes nature fascinating, creating rich forms such as high mountain peaks, low hills and hursts, deep valleys, wide plains and intricate coastlines. Can you imagine landscapes resembling the monotony of series produced by processes such as white noise or even Markovian? Now substitute time for space (or add time to space) and you move from topography to the evolution of natural processes. The combined long tails produce time series with strange shapes such as hockey sticks, peaks and plains, with rapid and also gradual changes, or phenomena such as persistent droughts (Joseph effect), severe floods (Noah effect), movement of continents, earthquakes, etc. Destructive and catastrophic? Yes, no doubt. But it depends on the point of view. On a narrow time scale and a local, perhaps egotistic, view these phenomena are destructive and catastrophic. On a broader time scale and a more global view the same phenomena become constructive and creative. These and others even more violent have formed our cosmos. Very interesting. I like the term “long tail” better than the more common in hydrology “heavy tail” and “fat tail” or the more technical “Pareto tail” and “power-law tail”, so I will adopt it from now on. I think “long tail” is more natural and expressive than the other names. For example, one says that a cat has a long tail (not a heavy tail). In addition, typologically it is closer to “long-term persistence” and “long-range dependence” (that is, long-tail autocorrelation).

Long tails in probability distribution and in autocorrelation function are two different things, both implying some scaling (power law) behaviour on the tails of these functions. In my couple of entropy papers in Hydrological Sciences Journal (2005) I used the names “state scaling” and “time scaling” for these two scaling types (the second could also be “space scaling” in a spatial process – if one substitutes space for time). Simultaneously, I tried to demonstrate in these that both long tails may be consequences of a single thing, the principle of maximum entropy, viz. maximum uncertainty.

The combined effect of long tails in probability and long tails in autocorrelation, that is the dominance of maximum entropy, makes nature fascinating, creating rich forms such as high mountain peaks, low hills and hursts, deep valleys, wide plains and intricate coastlines. Can you imagine landscapes resembling the monotony of series produced by processes such as white noise or even Markovian? Now substitute time for space (or add time to space) and you move from topography to the evolution of natural processes. The combined long tails produce time series with strange shapes such as hockey sticks, peaks and plains, with rapid and also gradual changes, or phenomena such as persistent droughts (Joseph effect), severe floods (Noah effect), movement of continents, earthquakes, etc. Destructive and catastrophic? Yes, no doubt. But it depends on the point of view. On a narrow time scale and a local, perhaps egotistic, view these phenomena are destructive and catastrophic. On a broader time scale and a more global view the same phenomena become constructive and creative. These and others even more violent have formed our cosmos.

]]>