A Comparison Of Methods For Solving MAX-SAT Problems

A Comparison of Methods for Solving MAX-SAT
Problems
Steve Joy, John E. Mitchell
Mathematical Sciences,
Rensselaer Polytechnic Institute,
Troy NY 12180 USA.
email: joys@rpi.edu, mitchj@rpi.edu
http://www.math.rpi.edu/~mitchj
Brian Borchers
Mathematical Sciences,
New Mexico Tech,
Socorro, NM 87801 USA
email: borchers@rpi.edu
http://www.nmtech.edu/~borchers
Seattle INFORMS Conference
October 25{28, 1998
Session MD29

Seattle INFORMS MD29 1
Abstract
We compare the performance of 3 approaches to the solu-
tion of MAX-SAT problems, including a version of the Davis-
Putnam-Loveland algorithm extended to solve MAX-SAT, an
integer programming branch-and-cut algorithm and an algo-
rithm for MAX-2-SAT problems based on a semide nite pro-
gramming relation.
1 Overview
De nition of MAXSAT.
Expressing MAXSAT as an integer program.
Cutting planes.
Extended Davis-Putnam-Loveland algorithm.
Semide nite programming relaxation of MAX2SAT.
Computational results.
Conclusions.

2 De nition of MAXSAT
A propositional logic formula F can always be written in
conjunctive normal form with m clauses and n vari-
ables.
Each clause is a disjunction of a set of literals.
Each literal is either a variable or the negation of a vari-
able.
The formula is the conjunction of the clauses.
For example:
F = (X1 _
X2) ^ (X1 _ X2 _
X3) ^ (X2 _ X3):
The satis ability problem, SAT:
Given a propositionallogic formulaF in conjunctive
normal form, is there an assignment of the logical
variables that makes F true?
The maximum satis ability problem, MAXSAT:
Given a propositional logic formula F in conjunc-
tive normal form, nd an assignment of the logical
variables that maximizes the number of clauses in
F that are true.

Computational complexity
SAT is NP-Complete.
Even if each clause has exactly three literals, SAT is still
NP-complete.
Some special cases are solvable in polynomial time, includ-
ing 2-SAT and HORN-SAT.
MAXSAT is NP-Hard even if each clause has only two
literals.
There isno polynomialtime approximationscheme(PTAS)
for MAXSAT unless P=NP.
It is possible to approximate MAXSAT to within a factor
of 1.32 and MAX2SAT to within a factor of 1.14 in poly-
nomial time, using a semide nite programming approach
(Goemans and Williamson, 1995).

3 An integer programming approach
Any weighted MAXSAT instance can be expressed as an
equivalent integer programming problem.
For example:
C1 = v1 _v2 weight 1
C2 = v1 _v2 _v3 weight 4
C3 = v1 _v2 weight 3
Equivalent to:
min w1 + 4w2 + 3w3
s:t: (1 ,x1) + (1 ,x2) + w1 1
x1 + x2 + x3 + w2 1
x1 + (1 ,x2) + w3 1
xi = 0 or 1; i = 1; : : : ; 3; wi = 0 or 1; i = 1; : : : ; 3
We call the added variables w1; : : : ; w3 weighted vari-
ables.

Branch-and-cut
Use a primal heuristic similar to GSAT (Selman et al.,
1992, 1996) or the heuristics of Gu (1992,1993,1994) to nd
a good solution.
Solve the linear programming relaxation obtained
by requiring 0 xi 1 for i = 1;:::;n, 0 wi 1 for
j = 1;:::;m.
If the solution to the relaxation is non-integral, modify it
by adding cutting planes, or branch.
Use three di erent types of cutting planes:
{ Resolution cuts (Hooker, 1988.)
{ Odd cycle inequalities (Cheriyan et al., 1996.).
{ Clique cuts.
The cuts are added locally, so we don't lift them to be
valid throughout the branch-and-cut tree.
Branching: split the relaxation into two problems, in one
of which a speci c variable is required to equal zero and in
the other it is required to equal one.
The algorithm is written in C. It uses MINTO to handle
the branch-and-cut tree, which uses CPLEX 4.0 to solve
the linear programming relaxations.

Clique cuts
Motivated bypigeonholeproblems(Hooker): holen asks
if it is possible to place n + 1 pigeons in n holes without
two pigeons being in the same hole.
We describe these cuts for 2-SAT problems:
Example: Assume our instance of 2SAT contains the four
literals l1; l2; l3; l4 and the six clauses:
l1 _ l2; l1 _ l3; l1 _ l4; l2 _ l3; l2 _ l4; l3 _ l4:
We can think of this as giving a clique, with vertices la-
belledbythe four literals, andedgesgivenbythe sixclauses.
Any satisfying truth assignment must have at least three
of these literals true. This gives a potential cutting plane.
Thiscanbeextended toMAX2SATandgeneral MAXSAT
by regarding some edges in the clique as missing and mod-
ifying the cutting plane appropriately. This is really a lift-
ing procedure.
We seek inequalities that are violated by at least 0.1. The
separation routine looks for literals with large values and
tries to nd cliques involvingsuch variables. The cliques are
built vertex-by-vertex. We reject inequalities that contain
too many weighted variables.

Resolution cuts
A classical logical procedure.
Also implemented in branch-and-cut by Hooker and Fedjki,
1990.
Example: Given two clauses, one containing a variable and
the other containing its complement:
x1 _ x3 _ x7 _ w9
x2 _ x3 _ x7 _ w11
we can resolve to generate the following:
x1 _ x2 _ x7 _ w9 _ w11
Hookershowedthat thisisequivalentto a Chvatal-Gomory
cutting planeprocedure in the integer programmingsetting.
Our resolution routine consists of a sequence of passes, each
designed to generate resolvents satisfying certain criteria.
Only accept resolved clauses that are unit clauses and
that are violated by at least 0.3.

Odd cycle inequalities
Derived by Cheriyan et al., 1996, motivated by similar in-
equalities for MAXCUT problems.
Example:
x1 +x2 +w10 +w11 1
+x2 ,x3 +w11 0
,x3 ,x4 +w12 ,1
,x1 ,x4 +w13 ,1
The rst, third, and fourth constraints are odd; the second
is even. Adding these constraints together,dividing by 2
and rounding coecients we obtain:
x2 ,x3 ,x4 + w10 + w11 + w12 + w13 0:
Use a modi ed version of Dijkstra's algorithm to nd cut-
ting planes.
Look for cuts violated by at least 0.3, and of length at
most 2.
Odd cycle cuts combine together clauses of length 2.
Thus, only e ective if the current relaxation containsa large
number of clauses of length 2, eg, MAX2SAT problems.
For problems with longer clauses, can only use these cuts
deep in the tree.

Branching strategy
After generating cuts and applying bounds, we branch if
there are still fractional variables in the LP solution.
Thebranchingschemewe useisa modi cationof theJeroslow-
Wang (1990) scheme.
Jeroslow-Wang examines the probability of satisfying all
the constraints if we randomly assign values to the remain-
ing un xed variables. It attempts to pick a variable whose
assignment will have the greatest change in this probability.
Our approach di ers from Jeroslow-Wang in that:
1. We give considerably less weight to singleton clauses.
2. We consider the e ect on both branches.

4 Extended Davis-Putnam-Loveland
Davis-Putnam-Loveland (1960,1978) is a classical branch-
ing and backtracking procedure for solving SAT.
Borchers andFurman (1995) extendedthisto solve MAXSAT
problems. (AlsodoneindependentlybyWallace andFreuder
(1996).)
We maintain an upper bound, ub, given by the number
of unsatis ed clauses in the best known solution.
We keep track of unsat, the number of clauses left unsat-
is ed by the current partial solution.
If unsat ub, backtrack from the current partial solu-
tion.
Written in C.
Code uses primal heuristic to nd good solution.
Incorporates unit clause trackingwhen unsat = ub,1.
Uses the variable selection rule of Dubois et al..

5 Semide nite programming
This formulation is due to Goemans and Williamson, 1995.
MAX2SAT as a QP
For each logical variable yi, introduce a 1 variable xi.
Add a variable x0. We'll set yi =TRUE if x0 = xi, and
FALSE otherwise.
Let
v(yk _yl) := 0:25(1 + x0xk + 1 + x0xl + 1 ,xkxl):
We obtain v(yk _ yl) = 0 if the clause is FALSE, and 1
otherwise.
Similar formulas can be constructed for clauses involving
negated literals.
The MAX2SAT problem becomes:
maxf
m
X
j=1
v(Cj) : xi 2 f,1; 1g; i = 0; : : : ; ng:

Formulation as an SDP
Express QP as:
maxfxT
Ax + c : xi 2 f,1;1g;i = 0;:::;ng
for a symmetric matrix A and a scalar c.
Note that xTAx = tr(XA), where X = xxT and tr de-
notes the trace of a matrix.
Rewrite problem as:
max tr(XA) + c
subject to X = xxT
xi 2 f,1;1g; i = 0;:::;n:
Notice that Xii = 1.
Relax the requirement that X be a rank 1 matrix to the
requirement that it be a positive semide nite matrix.
Write this X 0.
Obtain the semide nite programming (SDP) relaxation of
MAX2SAT:
max tr(XA) + c
subject to X 0
Xii = 1; i = 0;:::;n:
This relaxation can be solved in polynomial time using an
interior point method.

Cutting planes
Can strengthen the SDP relaxation by adding linear in-
equalities, as shown by Helmberg and Rendl (1998) for
MAXCUT.
Add constraints of the form:
Xij + Xjk + Xik ,1
Xij ,Xjk ,Xik ,1
,Xij + Xjk ,Xik ,1
,Xij ,Xjk + Xik ,1:
These constraints are derived by considering possible values
for xixj, xjxk, and xixk for 1 variables.
Add these constraints selectively, as cutting planes.

Our SDP implementation
Written in C.
Uses the primal-dual interior point semide nite program-
mingalgorithmof Helmberg, Rendl, Vanderbei, andWolkow-
icz (1996) to solve the SDP relaxations.
Uses a primal heuristic due to Goemans and Williamson,
1995.
Adds cutting planes of the type described above.

6 Computational results
MAX2SAT
-
6
CPU
Clauses
0 500 1000 2000 3000 4000
1 sec
1 min
10 min
1 hr
4 hrs
y
y
yy
y
y
y
y
y
y
y
y
y
y
y
y
}
}
}
~~
~
~
~
~

~

~
~
~
y Branch-and-cut
} EDPL
~ SDP (successful)
SDP (gap=1)
Log(CPU time) vs number of clauses for random 50
variable MAX-2-SAT problems

Results for 100 variable MAX-2-SAT problems
problem clauses unsat. B+C EDPL SDP
name clauses CPU nodes CPU backtracks CPU lower upper
p2180 5 180 0 0.62 1 0.1 1 366.85 180 180
p2180 8 180 1 4.96 1 0.7 2 1015.99 179 179
p2180 6 180 1 4.89 1 0.7 47 911.65 178 178
p2180 3 180 2 5.18 1 0.7 40 1417.08 178 178
p2180 9 180 2 5.23 1 0.7 34 2497.35 178 179
p2180 2 180 2 5.29 1 0.7 10 2152.11 178 179
p2180 1 180 3 4.99 1 0.7 162 2299.32 177 178
p2180 7 180 4 5.25 1 0.8 964 1322.69 176 176
p2180 4 180 4 5.19 1 0.9 1773 2315.90 176 177
p2180 10 180 4 5.31 1 0.9 1933 1106.76 176 176
p2300 2 300 13 7.18 1 2229.1 21334438 2307.46 287 288
p2300 3 300 13 7.50 1 2497.9 23764177 2399.92 287 288
p2300 4 300 14 7.29 1 4618.5 47629271 2202.70 286 287
p2300 9 300 15 7.51 1 8746.0 85235320 1220.04 285 285
p2300 1 300 15 8.18 1 10564.3 104553113 2086.05 285 286
p2300 10 300 15 8.24 1 12693.1 125325067 2292.71 285 286
p2300 5 300 15 11.35 3 29738.0 309999407 2263.76 285 287
p2300 6 300 17 9.67 1 69325.3 704246333 2200.03 283 284
p2300 8 300 17 9.42 1 Not Run Not Run 2123.03 283 284
p2300 7 300 20 32.33 11 Not Run Not Run 2266.63 280 281
p2400 3 400 25 13.36 1 Not Run Not Run 2097.99 375 376
p2400 7 400 26 30.69 11 Not Run Not Run 2127.00 374 375
p2400 6 400 27 18.70 3 Not Run Not Run 2134.13 373 374
p2400 4 400 28 13.58 1 Not Run Not Run 640.15 372 372
p2400 2 400 28 18.29 3 Not Run Not Run 2134.42 372 373
p2400 1 400 29 21.75 3 Not Run Not Run 2157.36 371 372
p2400 5 400 29 45.13 15 Not Run Not Run 2167.33 371 373
p2400 10 400 30 64.82 11 Not Run Not Run 2141.70 370 372
p2400 8 400 33 71.13 11 Not Run Not Run 2166.69 367 369
p2400 9 400 34 121.73 19 Not Run Not Run 2074.66 365 368

Results on dense MAX-2-SAT problems
variables clauses B+C SDP
CPU CPU lower upper
50 100 3.32 56.15 96 96
200 5.70 46.61 184 184
300 9.51 93.63 268 268
400 12.65 34.45 355 355
500 26.94 78.56 434 434
1000 64.76 18.75 838 838
1500 1933.55 250.39 1228 1229
2000 1350.11 84.56 1614 1614
2500 3378.69 348.00 2007 2008
3000 523.75 38.77 2419 2419
3500 10142.31 88.67 2783 2783
4000 5761.59 58.76 3169 3169
4500 16958.45 352.68 3540 3541
5000 37241.84 298.27 3933 3933
100 1000 22915.42 5153.65 857 859
1500 MEM 5321.79 1270 1272
2000 MEM 1549.69 1685 1685
2500 MEM 6422.84 2055 2060
150 1000 MEM 24445.65 1285 1297
1500 MEM 23274.42 1309 1310
2000 MEM 30934.79 1702 1711
2500 MEM 21335.99 2104 2112

MAX3SAT
-
6
CPU
Clauses
200 250 300 350 400 450 500
5 sec
1 min
10 min
1 hr
4 hrs
y
y
y
y
y
y
y
}
}
}
}
}
y Branch-and-cut
} EDPL
Log(CPU time) vs number of clauses for random 50
variable MAX3SAT problems

Results for 100 variable MAX-3-SAT problems
problem clauses unsat. B+C EDPL
name clauses CPU nodes CPU backtracks
o430 3 430 0 0.97 1 0.2 1
o430 4 430 0 1.08 1 0.1 1
o430 2 430 0 2.19 1 0.1 1
o430 1 430 1 169.68 543 1.4 320
o430 5 430 2 1429.61 7541 5.2 8834
o500 1 500 0 3.65 1 0.2 1
o500 3 500 2 983.06 3139 2.9 3345
o500 2 500 3 2908.52 7095 31.3 87816
o500 5 500 4 7927.12 16889 203.1 735535
o500 4 500 4 9873.80 18793 162.5 572489
o575 5 575 4 6488.93 8609 151.9 491192
o575 3 575 5 13342.57 17253 602.2 2191647
o575 1 575 6 MEM MEM 4370.9 18334787
o575 2 575 7 MEM MEM 10488.5 47346157

Steiner tree systems
Available from ORlib. Encoding due to Kautz et al., 1997.
problem variables clauses ave clause optimal B+C
name length value CPU nodes
steind2 1295 1765 1.31 220 86.35 105
steind3 1416 1885 1.25 1646 3.08 1
steind4 1499 2074 1.28 2044 4.44 1
steind5 1749 2646 1.34 3419 11.42 1
steind7 2045 2325 1.15 103 2.90 3
steind8 2166 2544 1.15 1180 3.14 1
steind9 2249 2797 1.20 1585 4.73 1
steind10 2499 3261 1.23 2219 10.46 1
steind11 5120 5882 1.17 29 467.52 333
steind12 5009 5039 1.01 42 2.57 1
steind13 5166 5499 1.06 544 4.29 1
steind14 5249 5709 1.08 740 5.81 1
steind15 5499 6202 1.11 1193 11.41 1
steind16 25032 25133 1.01 13 18.41 2
steind18 25166 25412 1.01 262 13.42 1
steind19 25249 25585 1.01 359 15.02 1
steind20 25499 26041 1.02 558 20.54 1
Results for Steiner D tree problems

Pigeonhole problems Hooker.
holen asks if it is possible to place n + 1 pigeons in n holes
without two pigeons being in the same hole.
-
6
CPU
Holes
0 5 10 15 20 25
1 sec
5 sec
30 sec
1 min
5 min
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
}
}
}
y Branch-and-cut
} EDPL
Log(CPU time) vs Number of Pigeons for pigeon
hole problems

Branch-and-cut on an instance
depth nodes resolution odd cycle clique
cuts/node cuts/node cuts/node
0 1 0.000 0.000 0.000
1 2 14.000 0.000 0.000
2 4 26.750 0.500 0.000
3 8 7.750 2.375 0.250
4 16 14.000 6.688 0.063
5 32 9.906 9.313 0.219
6 64 7.469 14.984 0.109
7 128 4.656 18.430 0.297
8 254 4.961 24.047 0.335
9 457 3.569 22.571 0.267
10 585 2.829 21.043 0.251
11 412 2.415 17.944 0.204
12 179 2.302 15.391 0.196
13 60 1.733 11.050 0.233
14 22 2.000 9.363 0.227
15 4 0.000 6.500 0.000
16 1 0.000 8.000 0.000
Cuts per node in the branch and cut tree of a 50
variable, 500 clause MAX3SAT instance

7 Conclusions
Branch-and-cut generates smaller trees than EDPL,
but spends more time at each node.
Branch-and-cut is very e ective for MAX2SAT prob-
lems with a reasonable number of clauses.
OncetheMAX2SATinstancehasalargenumber ofclauses,
the semide nite programming approach is superior.
The SDP relaxation appears to improve as the number of
clauses increases.
EDPL is superior for MAX3SAT. For MAX3SAT, the
branch-and-cut tree can expand in almost a binary fashion
until a sucient number of clauses of length 2 have been
generated.
Branch-and-cut is superior for the Steiner problems
and for the Pigeonhole problems.
The clique inequalities are essential for e ective solution
of the Pigeonhole problems.

References
[1] B. Borchers and J. Furman. A two-phase exact algorithm for MAX-SAT and
weighted MAX-SAT problems. Technical report, Mathematics Department, New
MexicoTech, Socorro, NM87801, October 1995. Revised: March1997. To appear
in Journal of Combinatorial Optimization.
[2] J. Cheriyan, W. H. Cunningham, L. Tun
cel, and Y. Wang. A linear programming
and rounding approach to max 2-sat. In David S. Johnson and Michael A. Trick,
editors, Cliques, Coloring and Satis ability: Second DIMACS Implementation
Challenge, DIMACS Series In Discrete Mathematics and Theoretical Computer
Science, volume 26, pages 395{414. AMS, 1996.
[3] M. Davis and H. Putnam. A computing procedure for quanti cation theory. J.
Assoc. Comput. Mach., 7:201{215, 1960.
[4] Michel X. Goemans and David P. Williamson. Improved Approximation Algo-
rithms for Maximum Cut and Satis ability Problems Using Semide nite Pro-
gramming. J. Assoc. Comput. Mach., 42:1115{1145, 1995.
[5] J. Gu. Ecient local search for very large-scale satis ability problems. SIGART
Bulletin, 3, 1992.
[6] J. Gu. Local search for satis ability (sat) problem. IEEE Transactions On
Systems Man And Cybernetics, 23:1108{1129, 1993.
[7] J. Gu. Global optimization for satis ability (sat) problem. IEEE Transactions
On Knowledge and Data Engineering, 6:361{381, 1994.
[8] J. Gu, P.W. Purdom, J. Franco, and B.W. Wah. Algorithms for the satis ability
(SAT) problem: A survey. In D. Du, J. Gu, and P. Pardalos, editors, Satis ability
Problem: Theory and Applications, volume 35 of DIMACS Series in Discrete
Mathematics and Theoretical Computer Science, pages 19{151. AMS/DIMACS,
1997.
[9] C. Helmberg and F. Rendl. Solving quadratic (0,1)-problems by semide nite
programs and cutting planes. Mathematical Programming, 82:291{315, 1998.

[10] C. Helmberg, F. Rendl, R. J. Vanderbei, and H. Wolkowicz. An interior point
method for semide nite programming. SIAM Journal on Optimization, 6:342{
361, 1996.
[11] J. N. Hooker. A quantitive approach to logical inference. Decision Support
Systems, 4:45{69, 1988.
[12] J. N. Hooker and C. Fedjki. Branch-and-cut solution of inference problems in
propositional logic. Annals of Mathematics and Arti cial Intelligence, 1:123{139,
1990.
[13] R. G. Jeroslow and J. Wang. Solving propositional satis ability problems. Annals
of Mathematics and AI, 1:167{187, 1990.
[14] H. Kautz, B. Selman, and Y. Jiang. A general stochastic approach to solving
problems with hard and soft constraints. In D. Du, J. Gu, and P. Pardalos,
editors, Satis ability Problem: Theory and Applications, volume 35 of DIMACS
Series in Discrete Mathematics and Theoretical Computer Science, pages 573{
586. AMS/DIMACS, 1997.
[15] D. Loveland. Automated theorema proving: A logical basis. North-Holland, New
York, 1978.
[16] B. Selman, H. A. Kautz, and B. Cohen. Local search strategies for satis ability
testing. In David S. Johnson and Michael A. Trick, editors, Cliques, Coloring
and Satis ability: Second DIMACS Implementation Challenge, DIMACS Series
In Discrete Mathematics and Theoretical Computer Science, volume 26, pages
521{532. AMS, 1996.
[17] B. Selman, H. Levesque, and D. Mitchell. A new method for solving hard satis-
ability problems. In Proceedings of the Tenth National Conference on Arti cial
Intelligence (AAAI-91), Anaheim, CA, pages 440{446, July 1992.
[18] R. J. Wallace and E. C. Freuder. Comparative studies of constraint satisfaction
and davis-putnam algorithms for maximum satis ability problems. In David S.
Johnson and Michael A. Trick, editors, Cliques, Coloring and Satis ability: Sec-
ond DIMACS Implementation Challenge, DIMACS Series In Discrete Mathe-
matics and Theoretical Computer Science, volume26, pages 587{615. AMS,1996.

A Comparison Of Methods For Solving MAX-SAT Problems

More Related Content

Similar to A Comparison Of Methods For Solving MAX-SAT Problems (20)

More from Karla Adamson (20)

Recently uploaded (20)

A Comparison Of Methods For Solving MAX-SAT Problems