Parallel Artificial Bee Colony Algorithm
0 likes244 views
The document discusses solving the vehicle routing problem (VRP) using the artificial bee colony (ABC) algorithm, focusing on its implementation in parallel across multiple cores. It reviews several research papers to outline improvements such as the swap operator for neighborhood search and the design of a parallel implementation to mitigate data dependencies. Lastly, it presents performance results from strong and weak scaling tests on various problem sizes and highlights future work to address additional VRP variations.
![Research Paper Analysis
Research Paper 1 [1]
● Problem addressed:
This paper looks at the application of the ABC algorithm to the capacitated VRP
problem. Capacitated VRP adds further constraints to the traditional VRP problem
to make it more complex. These constraints can be vehicle capacity, vehicle
mileage, etc. This paper also talks about the modifications that the authors had to
make to the ABC algorithm to suit their needs.
● Approach:
The authors solve the problem by initially generating random valid solutions.
Additional resources are then assigned to these solutions to perform a neighborhood
search. Neighborhood search means generating more valid solutions by using the
swap operation. The algorithm keeps track of the best solution found, and generates
a new random solution if the neighborhood search fails (to improve the solution
quality). This whole process is repeated till a maximum limit is reached and the best
solution in memory is assigned as the output.
● Novel Contribution:
The novel contributions of this paper include the solution scheme representation.
The authors represent each solution as an array containing numbers representing
each customer. A vehicle’s route is understood to begin from the depot, represented
by the number 0, and each subsequent number is the customer that the vehicle
services in order before returning to the depot.
Solution Scheme Representation [1]
Another novel contribution of the paper is the swap operator. This operation allows
us to generate new solutions during the neighborhood search by swapping the order
that one or more customers are visited in. The idea here is that this may result in a
shorter path to service all customers.](https://image.slidesharecdn.com/finalreport-161227113036/85/Parallel-Artificial-Bee-Colony-Algorithm-2-320.jpg)
![Swap Operation [1]
● Significance to our project:
The main take away from this paper for our project was the swap operator which we
use in our algorithm for neighborhood search. The solution scheme representation is
also adopted in our algorithm. We also generalize the paper’s approach to fit our
VRP problem.
Research Paper 2 [2]
● Problem addressed:
This paper evaluates different approaches to parallelizing the Artificial Bee Colony
algorithm and the challenges associated with each of these approaches. In
particular, it discusses the sequential dependencies that arise because of the
neighborhood search, probability calculation, and the roulette wheel selection
phases of the algorithm. This is because all three phases require sharing and
synchronizing data from all running processes during the execution of the
algorithm.
● Approach:
The paper suggests employing a shared memory architecture for parallelizing the
ABC algorithm. This is to be done by storing all solutions in local memory with a
copy in the global shared memory. Each processor then generates a new solution in
each cycle and overwrites the previous solution if the new one is better. At the end
of all trials, the best solution amongst all the processors is selected as the final
solution.
● Novel Contribution:
The novel contribution of this paper is its design of a parallel implementation of the
ABC algorithm. The design presented minimizes the overhead for the neighborhood
search by dividing the potential solutions equally among the available processors.
The probability calculation and roulette wheel selection is then done locally within
each processor and the generated candidate solutions are stored in local memory.
Finally, the local solutions are reduced in parallel to select the best found solution.
● Significance to our project:
The significance of this research paper for our project lies in the suggested parallel
design pattern. We use the approach of computing the fitness function and roulette](https://image.slidesharecdn.com/finalreport-161227113036/85/Parallel-Artificial-Bee-Colony-Algorithm-3-320.jpg)
![wheel selection locally at each processor as suggested by the author and modify the
algorithm to suit our particular need of solving the VRP.
Research Paper 3 [3]
● Problem addressed:
This paper extends the vehicle routing problem by adding further constraints of
delivery time periods. This is also known as the Periodic Vehicle Routing Problem
(PVRP). So, in this modified problem, every vehicle sets out from the depot,
services all the customers scheduled for a particular time period, and returns back.
Each customer is associated with a demand and service frequency.
● Approach:
The PVRP requires the delivery routes to be optimized for each time period.
So, the problem can be reformulated as one of n-VRP wherein n is the number of
time periods. The VRP by itself is one of the toughest combinatorial problems
known, so having to solve n instances of it is far beyond the reach of conventional
algorithms. The authors aim to solve the problem using the Artificial Bee Colony
Algorithm by optimizing the local search phase of the algorithm.
● Novel Contribution:
The paper’s major contribution is an optimization procedure they propose for the
2-opt method. The 2-opt method itself improves the efficiency of the local search
phase of the ABC algorithm by identifying ‘crossover’ points in the path.
Traditionally, crossover points have been identified by an enumeration method that
requires going over all the paths between customer nodes. This is a potential area of
improvement for which the authors propose a scanning strategy that identifies
crossover points according to the angle of the customer with respect to the
distribution center.
● Significance to our project:
We haven’t implemented the optimization suggested by the authors for the 2-opt
method, since this would require a complete remodelling of how we generate and
process the solutions, which time would not permit. But it remains as a potential
area that can be explored in the future. Since, we have already laid the foundation
for the parallelization of our programs, these improvements to the local search
phase can be made independent of the overall design.](https://image.slidesharecdn.com/finalreport-161227113036/85/Parallel-Artificial-Bee-Colony-Algorithm-4-320.jpg)
![Artificial Bee Colony Algorithm Outline [1]](https://image.slidesharecdn.com/finalreport-161227113036/85/Parallel-Artificial-Bee-Colony-Algorithm-6-320.jpg)
![Parallel Program
The Artificial Bee Colony algorithm is tricky to parallelize because of the many sequential
dependencies inherent in the algorithm. However, following the parallel paradigm
suggested in [2], allows us to implement the algorithm in parallel over several processors.
This is done as outlined in the following steps:
1. For N rounds, do the following -
2. Generate SP random solutions, where S is total number of employed bees and P is
total number of processors.
3. Divide these initial solutions randomly amongst all processors
4. Within each processor, run employed phase of the ABC algorithm
5. Within each processor, run onlooker phase of the ABC algorithm
6. Goto step 4 if less than N rounds
7. Parallely reduce local best solutions
8. Select best local solution as final result
Developer’s Manual
The sequential and parallel versions of our program have been tested on the nessie parallel
machine of the RIT Computer Science department. To compile the programs, store all
Java source files in a new directory, and execute the following commands on the bash
shell:
$ export CLASSPATH=.:/var/tmp/parajava/pj2/pj2.jar
$ javac *.java](https://image.slidesharecdn.com/finalreport-161227113036/85/Parallel-Artificial-Bee-Colony-Algorithm-7-320.jpg)
Parallel Artificial Bee Colony Algorithm
- 1. Vehicle Routing using Artificial Bee Colony Algorithm Team Embarrassingly Parallel Ajinkya Dhaigude ad8454@rit.edu Sameer Raghuram sr3669@rit.edu Overview The Vehicle Routing Problem (VRP) is a combinatorial optimization problem where the goal is to find the most optimal set of routes for a fleet of vehicles to service a given set of customers. This can be seen as a general case of the Travelling Salesman Problem where there are m vehicles that start out of a depot, service each customer once, and then return back to the starting depot. Also, several further constraints can be applied to the problem statement such as the capacity of each vehicle, time windows for each customer, etc. We try to solve the Vehicle Routing Problem by using the Artificial Bee Colony (ABC) algorithm--an optimization algorithm that mimics the swarm intelligence of bees in nature. We implement this algorithm in parallel over several cores and present a comparative study of the results. Problem Description We solve the Vehicle Routing Problem for the general case with one depot and m vehicles servicing n customers. We represent this with a fully connected graph where the node with index 0 represents the depot and all other nodes represent a customer. These nodes are generated with random x and y coordinate values and the distance between any two nodes is computed as a euclidean distance. Our problem is to start at the depot and minimize the total distance travelled to visit each node exactly once and then return to the depot. This task is to be accomplished by using all the m vehicles at our disposal.
- 2. Research Paper Analysis Research Paper 1 [1] ● Problem addressed: This paper looks at the application of the ABC algorithm to the capacitated VRP problem. Capacitated VRP adds further constraints to the traditional VRP problem to make it more complex. These constraints can be vehicle capacity, vehicle mileage, etc. This paper also talks about the modifications that the authors had to make to the ABC algorithm to suit their needs. ● Approach: The authors solve the problem by initially generating random valid solutions. Additional resources are then assigned to these solutions to perform a neighborhood search. Neighborhood search means generating more valid solutions by using the swap operation. The algorithm keeps track of the best solution found, and generates a new random solution if the neighborhood search fails (to improve the solution quality). This whole process is repeated till a maximum limit is reached and the best solution in memory is assigned as the output. ● Novel Contribution: The novel contributions of this paper include the solution scheme representation. The authors represent each solution as an array containing numbers representing each customer. A vehicle’s route is understood to begin from the depot, represented by the number 0, and each subsequent number is the customer that the vehicle services in order before returning to the depot. Solution Scheme Representation [1] Another novel contribution of the paper is the swap operator. This operation allows us to generate new solutions during the neighborhood search by swapping the order that one or more customers are visited in. The idea here is that this may result in a shorter path to service all customers.
- 3. Swap Operation [1] ● Significance to our project: The main take away from this paper for our project was the swap operator which we use in our algorithm for neighborhood search. The solution scheme representation is also adopted in our algorithm. We also generalize the paper’s approach to fit our VRP problem. Research Paper 2 [2] ● Problem addressed: This paper evaluates different approaches to parallelizing the Artificial Bee Colony algorithm and the challenges associated with each of these approaches. In particular, it discusses the sequential dependencies that arise because of the neighborhood search, probability calculation, and the roulette wheel selection phases of the algorithm. This is because all three phases require sharing and synchronizing data from all running processes during the execution of the algorithm. ● Approach: The paper suggests employing a shared memory architecture for parallelizing the ABC algorithm. This is to be done by storing all solutions in local memory with a copy in the global shared memory. Each processor then generates a new solution in each cycle and overwrites the previous solution if the new one is better. At the end of all trials, the best solution amongst all the processors is selected as the final solution. ● Novel Contribution: The novel contribution of this paper is its design of a parallel implementation of the ABC algorithm. The design presented minimizes the overhead for the neighborhood search by dividing the potential solutions equally among the available processors. The probability calculation and roulette wheel selection is then done locally within each processor and the generated candidate solutions are stored in local memory. Finally, the local solutions are reduced in parallel to select the best found solution. ● Significance to our project: The significance of this research paper for our project lies in the suggested parallel design pattern. We use the approach of computing the fitness function and roulette
- 4. wheel selection locally at each processor as suggested by the author and modify the algorithm to suit our particular need of solving the VRP. Research Paper 3 [3] ● Problem addressed: This paper extends the vehicle routing problem by adding further constraints of delivery time periods. This is also known as the Periodic Vehicle Routing Problem (PVRP). So, in this modified problem, every vehicle sets out from the depot, services all the customers scheduled for a particular time period, and returns back. Each customer is associated with a demand and service frequency. ● Approach: The PVRP requires the delivery routes to be optimized for each time period. So, the problem can be reformulated as one of n-VRP wherein n is the number of time periods. The VRP by itself is one of the toughest combinatorial problems known, so having to solve n instances of it is far beyond the reach of conventional algorithms. The authors aim to solve the problem using the Artificial Bee Colony Algorithm by optimizing the local search phase of the algorithm. ● Novel Contribution: The paper’s major contribution is an optimization procedure they propose for the 2-opt method. The 2-opt method itself improves the efficiency of the local search phase of the ABC algorithm by identifying ‘crossover’ points in the path. Traditionally, crossover points have been identified by an enumeration method that requires going over all the paths between customer nodes. This is a potential area of improvement for which the authors propose a scanning strategy that identifies crossover points according to the angle of the customer with respect to the distribution center. ● Significance to our project: We haven’t implemented the optimization suggested by the authors for the 2-opt method, since this would require a complete remodelling of how we generate and process the solutions, which time would not permit. But it remains as a potential area that can be explored in the future. Since, we have already laid the foundation for the parallelization of our programs, these improvements to the local search phase can be made independent of the overall design.
- 5. Sequential Program The sequential version of the program implements the Artificial Bee Colony algorithm where initial random solutions are stored following the solution scheme discussed above. This is done by creating an array that contains all the customer indices, in order, and a depot index for each vehicle at the end. This array is then shuffled using the Fisher-Yates algorithm to generate a random solution. The fitness of a solution is computed as the total euclidean distance between the coordinates of each pair of subsequent nodes in the solution array. This algorithm is outlined as follows: 1. For N rounds, Do 2. Generate SN random solutions, where SN is total number of employed bees 3. Employed Phase: Neighborhood Search a. Generate candidate solutions by using the swap operator b. Evaluate fitness of new solutions c. Replace local best solution if better solution found 4. Onlooker Phase: Roulette Wheel Selection a. Discard exhausted solutions b. Generate a new local solution for each discarded solution c. For each local best solution, take its reciprocal d. Use roulette wheel selection to select solutions for further exploration 5. Goto step 3 if less than N rounds 6. Select best local solution as final result
- 6. Artificial Bee Colony Algorithm Outline [1]
- 7. Parallel Program The Artificial Bee Colony algorithm is tricky to parallelize because of the many sequential dependencies inherent in the algorithm. However, following the parallel paradigm suggested in [2], allows us to implement the algorithm in parallel over several processors. This is done as outlined in the following steps: 1. For N rounds, do the following - 2. Generate SP random solutions, where S is total number of employed bees and P is total number of processors. 3. Divide these initial solutions randomly amongst all processors 4. Within each processor, run employed phase of the ABC algorithm 5. Within each processor, run onlooker phase of the ABC algorithm 6. Goto step 4 if less than N rounds 7. Parallely reduce local best solutions 8. Select best local solution as final result Developer’s Manual The sequential and parallel versions of our program have been tested on the nessie parallel machine of the RIT Computer Science department. To compile the programs, store all Java source files in a new directory, and execute the following commands on the bash shell: $ export CLASSPATH=.:/var/tmp/parajava/pj2/pj2.jar $ javac *.java
- 8. User’s Manual Run the sequential version of the program with the command: $ java pj2 ABCSeq "RandomGraph(<nodes>,<range>,<seed>)" <V> <S> Run the parallel version of the program with the command: $ java pj2 cores=<cores> ABCSmp "RandomGraph(<nodes>,<range>,<seed>)" <V> <S> where: <nodes> = number of customers in the graph including the depot <range> = maximum x, y coordinate <seed> = seed for random number generation <V> = total number of vehicles <S> = total number of employed bees
- 9. Strong Scaling We tested our sequential and parallel versions of the program on the nessie parallel machine for strong scaling. We ran this test for problem sizes of 250, 500, 1000, 2000, and 3000. The table below shows the results obtained: Size K T(msec) Speedup Efficiency 250 Seq 8745 1 1 1 8926 0.98 0.98 2 4648 1.881 0.941 3 3251 2.69 0.897 4 2478 3.529 0.882 5 2031 4.306 0.861 6 1767 4.949 0.825 7 1532 5.708 0.815 8 1521 5.75 0.719 500 Seq 17652 1 1 1 17861 0.988 0.988 2 9147 1.93 0.965 3 6314 2.796 0.932 4 4790 3.685 0.921 5 3979 4.436 0.887 6 3353 5.265 0.878 7 2892 6.104 0.872 8 2574 6.858 0.857 1000 Seq 36358 1 1 1 36573 0.994 0.994 2 18018 2.018 1.009 3 12098 3.005 1.002 4 9216 3.945 0.986 5 7440 4.887 0.977 6 6505 5.589 0.932 7 5607 6.484 0.926 8 5481 6.633 0.829 2000 Seq 77957 1 1 1 78105 0.998 0.998
- 10. 2 36848 2.116 1.058 3 24197 3.222 1.074 4 18151 4.295 1.074 5 14585 5.345 1.069 6 12296 6.34 1.057 7 10907 7.147 1.021 8 9624 8.1 1.013 3000 Seq 125782 1 1 1 125805 0.9998 1 2 56954 2.2085 1.104 3 36903 3.4084 1.136 4 27422 4.5869 1.147 5 21923 5.7374 1.147 6 18310 6.8696 1.145 7 15803 7.9594 1.137 8 14045 8.9556 1.119 Strong Scaling Results
- 11. We can make a few observations from the strong scaling results. The efficiency and speedup degrade as expected when we increase the number of cores. This could be due to the fact that our program has a few sequential dependencies, but since this degradation can
- 12. only be observed for small test cases, we can conclude that this is more due to the overhead in setting up the parallel team threads. For larger test cases, we can observe more than ideal speedup. We believe this is temporary since we can already observe the speedup and efficiency start to plateau. If we increase the number of cores beyond 8, we would be able to observe a slight degradation in speedup. As to why we observe more than ideal speedup and efficiency, this could be due to a variety of reasons, but we believe that it is down to the time the sequential program spends blocked as the JVM garbage collector frees up memory in large test cases. Weak Scaling As with strong scaling, we tested our sequential and parallel versions of the program on the nessie parallel machine for weak scaling. We ran this test for problem sizes of 200, 400, 600, 800, and 1000. The table below shows the results obtained: Size K T(msec) Speedup Efficiency 200 seq 7016 1 1 200 1 7179 0.977 0.977 400 2 7328 0.98 1.915 600 3 7403 0.99 2.843 800 4 7484 0.989 3.75 1000 5 7586 0.987 4.624 1200 6 7539 1.006 5.584 1400 7 7648 0.986 6.422 1600 8 7940 0.963 7.069 400 seq 14068 1 1 400 1 14281 0.985 0.983 800 2 14477 0.972 1.939 1200 3 14506 0.97 2.902 1600 4 14516 0.969 3.867 2000 5 14624 0.962 4.798 2400 6 14639 0.961 5.751 2800 7 14779 0.952 6.646
- 13. 3200 8 14831 0.949 7.569 600 seq 21328 1 1 600 1 21518 0.991 0.991 1200 2 21715 0.982 1.964 1800 3 21855 0.976 2.928 2400 4 21811 0.978 3.911 3000 5 21934 0.972 4.862 3600 6 22142 0.963 5.779 4200 7 22216 0.96 6.72 4800 8 22242 0.959 7.671 800 seq 28740 1 1 800 1 28978 0.992 0.981 1600 2 29124 0.987 1.953 2400 3 29219 0.984 2.92 3200 4 29297 0.981 3.883 4000 5 29492 0.975 4.821 4800 6 29596 0.971 5.765 5600 7 29788 0.965 6.683 6400 8 29876 0.962 7.615 1000 seq 36454 1 1 1000 1 36573 0.997 0.972 2000 2 36843 0.989 1.93 3000 3 36903 0.988 2.89 4000 4 37083 0.983 3.834 5000 5 37124 0.982 4.788 6000 6 37475 0.973 5.691 7000 7 37633 0.969 6.612 8000 8 37844 0.963 7.514 Weak Scaling Results
- 15. The weak scaling results show ideal size up. The efficiency for the problem size of 200 actually increases as the number of cores increase but this behavior can be attributed to the small problem size. The weak scaling performance for the other tests run as expected. Future Work The Vehicle Routing Problem has many variations and several further constraints can be applied to the VRP to make it more complex. In the future, it would be interesting to evaluate our algorithm against these further constrained problems. This would also require slight modifications to our algorithm to suit the new problem statements. Our algorithm currently uses a brute force approach to improve solution quality by running the ABC algorithm several times and selecting the best solution found. In the future, we would like to explore novel mechanisms to improve the solution quality with reduced computational overhead.
- 16. Another future work pertains to extending our algorithm to parallelize over the GPU. Certain phases of the algorithm seem ideal for GPU acceleration but this would also bring in new overhead. Evaluation of this technique is an interesting prospect. Lessons Learned Developing this parallel software program taught us several valuable lessons. The most important of these was identifying the parts of the algorithm most ideal for parallelization. This was followed by considering and discussing the various parallel approaches available for our needs and choosing the best amongst them. This project also gave us the opportunity to explore the sphere of biologically inspired swarm algorithms in the form of the Artificial Bee Colony algorithm. These algorithms are often used to solve combinatorial optimization problems and designing our own software around the ABC algorithm was a good learning experience. Individual Contributions Both the team members worked in close collaboration throughout the entire project. Sameer Raghuram suggested solving the Vehicle Routing Problem and Ajinkya Dhaigude came up with the idea of using the Artificial Bee Colony algorithm to do so. Both the authors spent time individually researching academic papers and then discussed the findings to arrive at a common literature survey. Ajinkya Dhaigude implemented the sequential program, and by extension the Artificial Bee Colony algorithm. The parallel version was implemented by Sameer Raghuram. Both authors performed code reviews and collaborated via git in the process of making the application. Overall this project wouldn’t be what it is without the contributions made by each team member, hence it is a team effort.
- 17. References 1. S. Z. Zhang and C. K. M. Lee, "An Improved Artificial Bee Colony Algorithm for the Capacitated Vehicle Routing Problem," Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on, Kowloon, 2015, pp. 2124-2128 2. Harikrishna Narasimhan, "Parallel artificial bee colony (PABC) algorithm," Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, Coimbatore, 2009, pp. 306-311 3. Baozhen Yao, Ping Hu, Mingheng Zhang and Shuang Wan, “Artificial bee colony algorithm with scanning strategy for the periodic vehicle routing problem”,SIMULATION, Transactions of the Society for Modeling and Simulation International, 2013, pp. 762-770