Probability and Likelihood

1. Discrete Variables

In the case that random variables have finite values. When the values a random variable XX can have are x1,,xnx_1, \cdots, x_n, the probability at X=xiX = x_i is pip_i which is denoted by P(X=xi)=piP(X = x_i) = p_i for 1in1 \le i \le n. Moreover, the function which consists of these probabilities p1,,pnp_1, \cdots, p_n is called the probability mass function. For example, when rolling a dice, a random variable XX is a finite value and pi=16p_i = \frac{1}{6}. The probability mass function is constant as shown below.


2. Continuous Variables

Consider the situation we select the number 55 in [1,6][1, 6]. Unlike the case of rolling a dice, there are infinite numbers in this range, so the probatility of picking the number 55 is 1=0\frac{1}{\infty} = 0. Other than this situation, picking the other real numbers has the same probability, 00. Therefore, it is meaningless that we find the probability when we pick a number in a continuous range, so we should only consider the specific range. For example, when we pick a number in [1,6][1, 6], the probability that the number is in [4,5][4, 5] is 5461=15\frac{5 - 4}{6 - 1} = \frac{1}{5}. This kind of probability functions cannot be represented by the probability mass function, and another probability function which is called the probability density function should be introduced. The area of this function in the specific range represents the probability that the number in this range is picked.


As shown in the figure, the total area must be 15×(61)=1\frac{1}{5} \times (6 - 1) = 1 because it is the sum of all probabilites. Since the probability density function is continuous, the probability, which is the area in this case, can be calculated only in the specific range.

3. Likelihood


As above, the shapes of functions look similar, and the probability at mm when a random variable XX is discrete is highest. However, when a random variable XX is continuous, the probability at mm is 00. Although the events near mm are intuitively most likely happen, the probability is always 00. This is an awkward situation, so another conept should be introduced which is called likelihood. As shown in the figure above, the likelihood has the different meaning depending on whether a random variable XX is discrete or continuous. However, it always means the yy-axis value of the probability mass function or the probability density function. Note that the probability is the likelihood when XX is discrete, but is not when XX is continuous.



© 2022. All rights reserved.