![No Knowledge
(uninformed)
Lexical Distance
ALPHABET = ('a'..'z').to_a + [" "]!
!
def distance a, b!
return nil unless a.length == b.length!
a.chars.each_index.inject(0) do |sum, i|!
sum + char_distance(a[i], b[i])!
end!
end!
!
def char_distance ai, bi!
ia = ALPHABET.index(ai)!
ib = ALPHABET.index(bi)!
ia = (ia - ALPHABET.length).abs if ia > (ALPHABET.length/2)!
ib = (ib - ALPHABET.length).abs if ib > (ALPHABET.length/2)!
(ia - ib).abs!
end
"hello steve" #=> 21!
"hello mr t" #=> 34!
" ello world" #=> 6!
"iello world" #=> 1!
"hello world" #=> 0
(2) Candidate solution](https://image.slidesharecdn.com/diagnosingcancerwithci-slideshare-140609034250-phpapp01/85/Diagnosing-cancer-with-Computational-Intelligence-17-320.jpg)
![Selection
def select entities, population!
# Rank by fitness!
fitnesses = entities.map{ |e| e.fitness }!
sum = fitnesses.reduce(:+).to_f!
normalized_ranks = ranks.map{ |r| r.to_f/sum }!
!
# Calculate, form array of tuples to keep a reference to rank & entity!
tuples = []!
entities.each_with_index{ |e, i| tuples << [e, normalized_ranks[i]] }!
tuples.sort!{ |a,b| a[1] <=> b[1] }!
!
# Select probabilistically based on rank!
size = population!
selected = []!
while selected.length < size!
tuples.each_with_index do |tuple, index|!
if rand() < tuple[1]!
selected << tuples[index][0]!
end!
end!
end!
selected!
end!](https://image.slidesharecdn.com/diagnosingcancerwithci-slideshare-140609034250-phpapp01/85/Diagnosing-cancer-with-Computational-Intelligence-34-320.jpg)
![The Perceptron
f(net)
w1
w2
w3
w4
x1
x2
x3
x4
Activation strength
f(net) = f(x1w1 + x2w2 + x3w3 + x4w4)
f is an activation function:
[step, sigmoid, h tan, linear]
sigmoid:](https://image.slidesharecdn.com/diagnosingcancerwithci-slideshare-140609034250-phpapp01/85/Diagnosing-cancer-with-Computational-Intelligence-47-320.jpg)
![PSO Objects
Particle:
velocity
position
personal best position
inertia
social_weight
cognitive_weight
(dimensions)
Swarm:
[particle]
global best position
floats](https://image.slidesharecdn.com/diagnosingcancerwithci-slideshare-140609034250-phpapp01/85/Diagnosing-cancer-with-Computational-Intelligence-58-320.jpg)
![Ones Problem
Use a PSO to generate a
vector of float values where
each value equals exactly 1
The solution: [1.0, 1.0, 1.0, 1.0, 1.0]](https://image.slidesharecdn.com/diagnosingcancerwithci-slideshare-140609034250-phpapp01/85/Diagnosing-cancer-with-Computational-Intelligence-61-320.jpg)
Diagnosing cancer with Computational Intelligence
- 1. Welcome to Jozi ruby
- 2. (Building for the web) (AI Fanatic) (Community support)
- 3. This talk is about CI, namely, Computational Intelligence – the study of adaptive mechanisms to enable or facilitate intelligent behaviour in complex and changing environments. These mechanisms include those CI paradigms that exhibit an ability to learn or adapt to new situations, to generalise, abstract, discover and associate.
- 5. AI Deterministic Stochastic Weak AI “specific” Strong AI “general” New era AI “CI” Old era AI “brute force”
- 8. 1. Search Space ‘fitness landscape’ 2. Candidate Solution 3. Fitness Function (heuristic) 0. Global Search concepts
- 10. The Problem Write a program that will print “hello world” to standard out in these four cases: ! 1. Deterministic algorithm with knowledge (informed) 2. Stochastic algorithm with knowledge (informed) 3. Deterministic algorithm with no knowledge (uninformed) 4. Stochastic algorithm with no knowledge (uninformed)
- 11. Deterministic & Knowledge puts "hello world"! 1
- 12. Stochastic & Knowledge if rand() < 0.95 # 5% failure rate! puts "hello world"! else! fail "a statistically unlikely death"! end! 2
- 13. No Knowledge Write a program that will print something to standard out, without knowing exactly what it is that should be outputted… (uninformed)
- 15. No Knowledge (uninformed) Define: (3) heuristic Computing proceeding to a solution by trial and error or by rules that are only loosely defined.
- 16. No Knowledge (uninformed) Define: (3) fitness function A single real value that reflects to some accuracy how close a solution is from being correct Sample solutions "hello steve" #=> half way there! "hello Mr. T" #=> half way there! " ello world" #=> almost there!!
- 17. No Knowledge (uninformed) Lexical Distance ALPHABET = ('a'..'z').to_a + [" "]! ! def distance a, b! return nil unless a.length == b.length! a.chars.each_index.inject(0) do |sum, i|! sum + char_distance(a[i], b[i])! end! end! ! def char_distance ai, bi! ia = ALPHABET.index(ai)! ib = ALPHABET.index(bi)! ia = (ia - ALPHABET.length).abs if ia > (ALPHABET.length/2)! ib = (ib - ALPHABET.length).abs if ib > (ALPHABET.length/2)! (ia - ib).abs! end "hello steve" #=> 21! "hello mr t" #=> 34! " ello world" #=> 6! "iello world" #=> 1! "hello world" #=> 0 (2) Candidate solution
- 18. (x,y,z) Deterministic & No Knowledge3 xy z (‘h’,’e’,’l’) ’e’ ’h’ ’l’ (1) Search space
- 19. Deterministic & No Knowledge3 “Arrow solution” require_relative 'distance'! # D = domain (all possible values)! D = ALPHA! # "hello world".length = 11 chars! best = "aaaaaaaaaaa"! D.each do |a|! D.each do |b|! D.each do |c|! D.each do |d|! D.each do |e|! D.each do |f|! D.each do |g|! D.each do |h|! D.each do |i|! D.each do |j| ##! D.each do |k| ###! candidate = "#{a}#{b}#{c}#{d}#{e}#{f}#{g}#{h}#{i}#{j}#{k}"! best = distance(candidate) < distance(best) ? candidate : best! puts "solution = #{best}" and exit 0 if distance(best) == 0! end ###! end ##! end! end! end! end! end! end! end! end! end!
- 20. Deterministic & No Knowledge3 Problem: takes far too long for a simple hello world, imagine something that required actual computing power… Combinations = 27^11 = 5.5590606e+15 for brute force ! Combinations = 27*11 = 297 for optimising each dimension independently (since the problem is ‘separable’)
- 21. Stochastic & No Knowledge4 Clearly, we need an intelligent solution… (0) Global Search!!!!!!!!!!!!!!!!!
- 22. RECAP 1. Search Space ‘fitness landscape’ 2. Candidate Solution 3. Fitness Function (heuristic) 0. Global Search concepts
- 23. Evolving a solution to hello world with natural selection
- 25. “The sequencing is just enormously complex” ~ Kevin “But, didn’t you write it?” ~ Sam “Ha, some of it. The rest is just, beyond me” ~ Kevin
- 27. GA: “Genetic Algorithm” EC Algorithmic model developed to simulate biological evolution.
- 28. Jean-Baptiste Lamarck’s theory of evolution was that of heredity, i.e. the inheritance of acquired traits. The main idea is that individuals adapt during their lifetimes, and transmit their traits to their offspring
- 29. Charles Darwin ~ Individuals with the “best” characteristics (traits/genes) are more likely to survive and to reproduce, and those characteristics will be passed on to their offspring. These desirable characteristics are inherited by the following generations, and (over time) become dominant among the population.
- 30. Genetic Algorithm Overview “hllew ordjg” “lpainbsy is” “heldnn hjao” “gn oojafh o” “woolnhloea ” Population “hllew ordjg” “lpainbsy is” “heldnn hjao” “gn oojafh o” “woolnhloea ” recom bine “hllewgwooln” “lpan hjy is” “hnbsoojaeldn” “gn aoifh o” “ ordjhloea ” “hllew ordjg” “lpainbsy is” “heldnn hjao” “gn oojafh o” “woolnhloea ” “holfwgxoono” “lpam ijy is” “hnarnojaeoen” “gn aoifh o” “ oqejhloez ” m utate “heldnn hjao” “woolnhloea ” “holfwgxoono” “hnarnojaeoen” “gn aoifh o” selection
- 31. Genetic Algorithm Overview @individuals = array_of_individuals! generations.times do |generation|! # Calculate fitness! @individuals.each{ |individual| individual.calc_fitness }! ! # Recombine (copulate)! offspring = @crossover_strategy.crossover(@individuals)! ! # Mutate offspring! mutated_offspring = @mutation_strategy.mutate(offspring, problem)! ! # Select next generation! generation_pool = (@individuals + mutated_offspring)! new_population = @selection_strategy.select(generation_pool, @population)! ! # Ensure elitism! @individuals = elitism(new_population, generation_pool)! end! ! return best_solution_found! !
- 32. Crossover / Recombination h e n n o x o r d l 0 1 1 0 1 0 0 0 1 1 0 g n l b y b w r o l d h n l n y x o o l l + = + = “Heredity”
- 33. Mutation h e n n o x o r d l 0 1 -1 0 1 0 0 0 -1 -1 0 + = h f m n p x o q c l “Diversity”
- 34. Selection def select entities, population! # Rank by fitness! fitnesses = entities.map{ |e| e.fitness }! sum = fitnesses.reduce(:+).to_f! normalized_ranks = ranks.map{ |r| r.to_f/sum }! ! # Calculate, form array of tuples to keep a reference to rank & entity! tuples = []! entities.each_with_index{ |e, i| tuples << [e, normalized_ranks[i]] }! tuples.sort!{ |a,b| a[1] <=> b[1] }! ! # Select probabilistically based on rank! size = population! selected = []! while selected.length < size! tuples.each_with_index do |tuple, index|! if rand() < tuple[1]! selected << tuples[index][0]! end! end! end! selected! end!
- 35. Elitism badassoftheweek.com Make sure the best individual(s) survive to the next generation
- 37. Diagnosing Cancer with NN & PSO
- 39. ~10b neurons each connected to thousands others via synapse
- 43. NN Neural Networks FFNN: “Feed Forward Neural Network” Artificial model developed to approximate the generalization of knowledge & discovery.
- 44. Classification Problem Attr1 Attr2 … AttrN Class 17.99 10.38 1.78 B 0.5 595.9 0.03 M 122.8 103.2 9.2 M 9.34 90 2.5 B … Input vector “Relevant data” Classfication “Target vector” Each row is 1 persons tissue measurements
- 45. Classification Problem Training 17.99 10.38 1.78 B B Output Target 0.5 595.9 0.03 B M 122.8 103.2 9.2 B M 9.34 90 2.5 M B 7.69 1.38 2.33 B M
- 46. Classification Problem Evaluation Attr1 Attr2 … AttrN 17.99 10.38 1.78 (Malignant or Benign)FNN: Rn -> {M,B} Class B
- 47. The Perceptron f(net) w1 w2 w3 w4 x1 x2 x3 x4 Activation strength f(net) = f(x1w1 + x2w2 + x3w3 + x4w4) f is an activation function: [step, sigmoid, h tan, linear] sigmoid:
- 48. The Perceptron: OR f(net) w1 w2 x1 x2 x x f(net) 0 0 0 0 1 1 1 0 1 1 1 1 Guess values for x1 and x2… (Step activation) f(net) = f(x1w1 + x2w2 + x3w3 + x4w4)
- 49. The Perceptron: OR & XOR x x f(net) 0 0 0 0 1 1 1 0 1 1 1 1 x x f(net) 0 0 0 0 1 1 1 0 1 1 1 0 1 0 0 1 1 0 0 1 OR: XOR:
- 50. The 3 Layer FFNN We can compose Perceptrons (which have activation functions) to create a higher order function from them that has more “information capacity”.
- 51. The 3 Layer FFNN Output FNN: Rn -> {M,B} ! meaning… ! output = fout(net) = fout(sum(wfmiddle(net))) = fout(sum(wfmiddle(sum(vz)))) w vz error = patterns.each do |pattern| sum(difference(target,output)) end / patterns.length Output Fitness (accuracy)
- 52. How do we train it? 1. Gradient descent 2. Simulated annealing 3. RPROP 4. Global search? PSO!
- 53. Training our NN with A PSO
- 55. Simple individuals, working socially to exhibit complex, emergent, behaviour
- 57. SI Swarm Intelligence ! PSO: “Particle Swarm Optimizer” Algorithmic model developed to simulate complex emergent behaviour of swarms.
- 58. PSO Objects Particle: velocity position personal best position inertia social_weight cognitive_weight (dimensions) Swarm: [particle] global best position floats
- 59. PSO Algorithm initialize_swarm! @iterations.times do |i|! @swarm.each do |particle|! particle.update_velocity! particle.update_position! particle.update_pbest! update_gbest particle! end! end
- 60. PSO Algorithm: Velocity & Position Updates Position update position = position + v Velocity update v = wv + c1r1(pbest - position) + c2r2(gbest - position)
- 61. Ones Problem Use a PSO to generate a vector of float values where each value equals exactly 1 The solution: [1.0, 1.0, 1.0, 1.0, 1.0]
- 62. DEMO Questions
- 63. Bringing it together Swarm Update personal best How well does it classify? set solution vector as weights in NN report fitness to update gbest in the swarm
- 64. DEMO Questions
- 65. fin.