The difference between playing against a human and a natural system is that the human can be accommodating or adversarial, while (we may assume) the natural system is indifferent to our success. The human can make the game same very hard or very easy depending on the clues, for example by selecting typical or atypical examples.

In this framework where all information are simply ‘clues’ the statement that “statistical significance is meaningless when discussing poorly understood systems” seems an odd aversion. All we have are clues, whether they are gained from macroscopic or microscopic probes. And we need all the clues we can get from all angles. I think the quote is actually talking about jumping to conclusions.

Like the adversary, the learner has skills that may include observation and experiment. Capacity to manipulate the system adds power to our probing. You can construct theoretical experimenters that are capable of an infinite number of experiments in a finite time, and they are capable of overcoming the Popperian limitation to falsification, and can prove universal statements (theories).

This framework came out of machine learning developed by Clark Glymour, including hierarchies of learners, such as the pantheon of Greek Gods, each being capable of more powerful discoveries. I see from a Google search his recent book is on the Android Mind, looking at the powers of computer minds for discovery.

” … for the editors of Thinking about Android Epistemology, there should be theories about other sorts of minds, other ways that physical systems can be organized to produce knowledge and competence.”

References:

Causation, Prediction, and Search – 2nd Edition
Peter Spirtes, Clark Glymour and Richard Scheines

In my view, understanding is not identical to making a deterministic (e.g. mechanistic) conceptualization of the system. A good example is statistical physics, which offers, in my opinion, better understanding of what happens in a litre of gas than a mechanistic description or a classical thermodynamical description do. For example, if one tries to describe this litre as a deterministic system of several molecules, I do not think one will derive any result for the system . But it is important to understand that the macroscopic behaviour is related to microscopic movement of molecules and infer the macroscopic behaviour using probability, entopy (which is a probabilistic concept), etc.

In another example, certainly each leaf of any tree plays a role in a the hydrological cycle of a catchment. If one would try to estimate the evaporation in the catchment analyzing each tree and each of its leaves separately, certainly ploughs the sand. Here probability would help to acquire a macroscopic picture of the catchment evaporation without the need to examine each leaf. But on the other hand, if one does not care about the process of evaporation at all – because perhaps he/she uses for instance a neural network whose input and ouput are merely rainfall and river flow, perhaps he/she will not acquire any understanding. In the latter case it would be impossible to do extrapolation, for example, for cases that are not represented in the data (e.g. extreme floods).

So, my view is that concepts such as probability and entropy do provide understanding for complex systems (allowing kind of integration of microscopic behaviours into macroscopic ones) whereas black box neural networks may not do.

Perhaps I am not understanding what you mean by understanding, but your views here seem to run counter to your work which seems to me to be strongly based in observations, and induction (if that is the right term) of system character from the observations.

I am not saying any approach leads to definitive answers. All approaches have limitations, and finding those limits is one of most worthwhile things you can do I think. And whether they are neural nets or whatever, the proof is in the rigorous validation, framing severe tests, making ‘surprising’ predictions, and not tautological ones. I don’t think it is the intent of neural nets to provide insight – they are black boxes by design. There main purpose is to predict (or more precisely, fit). Surely when you praise the old concepts of probability and statistics you include significance estimates (i.e. probability of events), or are you suggesting a statistics without significance testing?

I fact, I really don’t know what you mean by ‘understand a system’. You can break it into parts, but then it isn’t a system. The way you break it up is arbitrary and introduces artifacts. You can look for causation, but that is a philosophical black hole. Only statements like ‘minimizes some objective function’, like certainty, or satisfies some equality, like energy conservation, have a sense of meaning to me when talking about systems, and simple concepts like probability distributions.

“From a practical standpoint, however, it may be preferable to acknowledge that the concept of statistical significance is meaningless when discussing poorly understood systems.”

This may mean that first we should acquire some understanding of the system and then formulate models and test hypotheses. In turn, this may be contrary to the modern trend of “data-driven” models such as so-called “artificial neural networks” (which is another bad term in my opinion – there is nothing “neural” in these) and “attractor reconstructions” from data. Such tools may be good for very simple nonlinear systems but may fail to provide any insight in complex natural systems. In constrast, the old concepts of probability and statistics at least provide indications of the uncertainty and the limitations in modelling.