Probability in Computer Science

Probability is the science of uncertainty —and in computer science, uncertainty is everywhere. Whether it’s predicting what movie you’ll watch next, how secure your password is, or how long an algorithm might take to run, probability plays a key role. It helps computers make smart guesses, deal with randomness, and perform better in the real world.

It plays a crucial role in numerous fields, providing a framework for making informed decisions in uncertain situations.

Uses of Probability in Computer Science

There are various applications of probability in real life that include:

• Machine Learning: Many machine learning methods use probability to make predictions and decisions based on data.Example: The Naive Bayes Classifier uses conditional probability to classify data.
• Artificial Intelligence (AI): Decision Making under Uncertainty: AI systems often deal with uncertain situations. They use probability to make smart decisions. For example: Self-driving cars use probabilistic models to guess what nearby vehicles or pedestrians might do next.
• Natural Language Processing (NLP): Probability is important in language-related tasks like speech recognition, translation, and text generation. For Example :Tools like Markov Chains and HMMs help understand and predict sequences of words or sounds.
• Randomized Algorithms: Some algorithms use random choices to solve problems faster or more effectively. For example:Monte Carlo and Las Vegas Algorithms, Randomized Quicksort, and Simulated Annealing.
• Search Engines (Ranking Algorithms): Search engines like Google use probability to rank pages. For Example: The PageRank algorithm estimates the chance that a user clicks on a particular page.
• Computer Vision: Probabilistic methods are applied to interpret and process visual data from images or videos. For example: Object Recognition, Image Segmentation.
• Online Betting: Online betting platforms use probability to set odds and predict outcomes. They use algorithms and past data to calculate the chances of winning.
• Cybersecurity: In cybersecurity, probability helps predict possible attacks and assess risks. This supports the design of safer systems.

Through these applications, probability theory demonstrates its versatility and importance in navigating uncertainty and optimizing outcomes in various professional and everyday contexts.

Probability Concepts in Computer Science

Conditional Probability and Bayes' Theorem

Conditional Probability is the probability of an event A occurring, given that another event B has already occurred. Given by:

P(A∣B) =\frac{ P(A∩B)}{P(B)}

• P(A∣B) = Probability of A given B
• P(A∩B)​ = Probability that both A and B will occur
• P(B) = Probability of event B occurring.

Bayes' Theorem states that given P(B)>0, the probability of A given B is denoted by:

P(A∣B) =\frac{ P(B∣A) ×P(A)}{P(B)}

Application of Conditional Probability and Theorem in CS

• Naive Bayes Classifier/ Bayesian Classifiers

The Naive Bayes Classifier is a probability-based machine learning algorithm used for classification tasks. It applies Bayes’ Theorem to predict the class of a data point, assuming that all features are independent of each other (this is the "naive" assumption).
Compute the prior probability of each class. -> Compute the likelihood of each events -> Using Bayes Theorem estimate posterior probability(updated probability of an event) -> Selecting the class with the highest probability

Random Variables

A random variable is a function that assigns a numerical value to each outcome in the sample space of a random experiment. This helps us to quantify randomness mathematically.

For a random variable x, PDF, PMF, and CDF.

• PMF( Probability Mass Function) is calculated for Discrete Random Variables. This gives each value of x of the random variable X.
  p(x) = P(X=x)
• PDF(Probability Density Function) is calculated for a Continuous Random variable. This is found by integrating probabilities over the intervals.
  P(a \leq X \leq b) = \int_{a}^{b} f(x) \, dx
• CDF(Cumulative Distribution Function) gives the total probability up to a point. i.e for F(x) up to the point x.
  F(x)=P(X≤x)

Application of Random Variables in CS

• Randomized Quicksort: Models the pivot position and number of comparisons. Used to analyze: Expected runtime (average case) and distribution of recursion depth.
• Monte Carlo Algorithms: Monte Carlo Simulation is an algorithm to determine the likelihood of a range of results of occurring by using repeated random sampling.
• Las Vegas Algorithms: Models the runtime of the algorithm. Used to analyze:Expected runtime.

Expectation and Variance

Expectation (also called expected value) is the average or mean value you would expect from a random experiment if you repeat it many times.

Variance measures how much the values of a random variable spread out from its average.

Application of Expectation and Variance in Probability

• Modeling and Analysis: Expectation defines the average prediction or parameter estimate. Variance measures the uncertainty or spread of predictions.
• Analyzing Randomized Algorithms: Expectation is used to find the average running time or average cost of an algorithm that uses randomness. Variance helps understand the stability of the algorithm's performance.
• Randomized Load Balancing: Expectation predicts the average load on servers. Variance measures uneven load distribution (important to avoid bottlenecks).