Disruptive platform technologies usually have a broad base of application. During early stage development, before there is a developed market, the selection of a particular product is usually a ‘high risk, low data’ decision. There are a large number of unknowns, both the known unknowns and the unknown unknowns; we seek the resolve these over time. In this type of situation it is difficult to make the initial portfolio selection decision and to effectively monitor the resolution of uncertainty, and determine the ultimate ‘chance of success’ for the product.
Problems in portfolio selection and project monitoring
The portfolio selection process, even when highly structured, often reduces to persuasion by advocates and champions. When a lot of data is being presented it is easy to forget ‘how we arrived’ at a particular position, assigning a higher importance to things that we heard recently (or long ago, depending on how your mind works). Soaring rhetoric can outweigh sober analysis and dispassionate appraisal of risk. It can be difficult to judge the ‘quality’ of a piece of information, which may find itself as a lynchpin in an argument to take a particular course of action. With a lot of unknowns it can be difficult to formulate go/no-go metrics and not relax the criteria when you get to the decision point.
The field of behavior economics examines some of the less rational beliefs of Homo economicus. Work by Tversky and Kahneman illustrate cases of overconfidence in our abilities, the desire to go with the herd and a propensity for rolling rationalization. Here is a list of cognitive biases that you can easily imagine arise in portfolio selection processes.
Develop a simple methodology and toolset that allows us to :-
Reduce complex business decisions to specific and testable hypothesis, which can be definitively refuted.
Systematically revise our ‘belief’ in a hypothesis as we receive new information.
Integrate new information of many types and forms, of varying degrees of ‘quality’.
Maintain a history of how we arrived at a particular belief to provide an ‘audit trial’ or ‘memory’ to support future decisions and actions.
Integrate and logically connect hypothesis to create a ‘belief network’ that supports complex decision making.
Avoid cognitive biases and increase objectivity
Logic and Probability
There are three main modes of argument, deduction, induction and abduction (inference to best explanation IBE). Inductive logic analyses risky arguments using probability ideas. There are however different interpretations of what ‘a probability is’.
Frequentists talk about probabilities only when dealing with experiments that are random and well-defined. The probability of a random event denotes the relative frequency of occurrence of an experiment’s outcome, when repeating the experiment. Frequentists consider probability to be the relative frequency “in the long run” of outcomes.
Bayesians, however, assign probabilities to any statement whatsoever, even when no random process is involved. Probability, for a Bayesian, is a way to represent an individual’s degree of belief in a statement, given the evidence.
Logical Probability is thought of as a logical relation between a hypothesis and the evidence for it. J.M. Keynes and Rudolf Carnap both favored a logical theory of probability. Personal probabilities are a private matter, they are up to the individual and anything goes so long as be basic rules of coherency are obeyed. Logical probability maintains that there are uniquely correct, uniquely rational judgments of the probability of a hypothesis in the light of evidence.
For the purposes of decision making in a business context there are very few cases where a Frequentists approach can be used. We tend to use the Bayesian notion of probability where belief allows us to make investment decisions.
It is plausible to connect personal degrees of belief and personal betting rates
You would not pay more than $1 to win $2 on the flip of a coin. If you have some domain specific business knowledge that allows you to exploit an opportunity, your betting rate would be markedly different from someone without that knowledge. During product development as uncertainty is resolved our beliefs are updated and we revise the level of investment we are willing to make. People have always used this ‘managerial flexibility’ and there is now a move to formalize this type of ‘real option’ thinking in investment and portfolio selection.
Verificationism and Falsifiability
There are two common problems in portfolio decision making, how do we extrapolate experience to the future? And how can we provide definitive go/no-go criteria when we do not know the problem well? The former is the problem of induction, and is the question of whether inductive reasoning leads to truth. That is, what is the justification for presupposing that a sequence of events in the future will occur as it always has in the past (for example, that the laws of physics will hold as they have always been observed to hold). If we cannot assume uniformity of nature for physical laws we definitely cannot do so in a business context where we know that the landscape changes very quickly.
Often a go/no-go criteria is framed in a way that allows it to get out of jail down the line. A criteria such as, “show interest from a customer” is quite broad. If in a month’s time if we hear a statement “Fred and Jeff seem quite interested”, this adds practically no new useful knowledge upon which to base a decision – “A difference that makes no difference is no difference”. It also allows us to introduce an ad hoc revisions to ‘pass’ the criteria. If we set criteria such as “one sale made by the end of the quarter”, then we have something that is definitively testable. This is a criterion that puts itself at risk, which can be refuted or falsified – falsification adds new knowledge as it allows us to eliminate options and make definite investment decisions i.e. don’t invest. Falsifiability was put forward as solution to the problem of induction by Karl Popper.
This is related to the Logical Positivist view of the verifiability theory of meaning: the meaning of a sentence consists in its method of verification. In other words, if a sentence or statement has no possible method of verification, it has no meaning. It is pointless to make a go/no-go goal such as, “demonstrate our value proposition and facilitate end to end knowledge transfer”, as there is no possible way to test this and it therefore falls into the category of a nonsensical statement (also known as bullshit bingo).