The mean and median of a probability distribution coincide when the distribution is symmetric, making an indifferent choice between the two. In contrast, when the distribution is skewed or contains outliers, the median is preferred as it is unaffected by extreme values. Thus, the choice depends on the shape of the distribution.