![Solver Network
• Embedding function ℰℬ:
• multi-layer recursive embedding with interleaving forward and
reversed layers
• Last layer is a reversed layer so we can read the output from
Variable nodes.
• Output space encode the soft assignment (in range [0−1]) to the
corresponding variable node in the input circuit.
• ℱ 𝜃 acts as policy network.
Pooling: Retrieve only the sink nodes](https://image.slidesharecdn.com/learningtosolvecircuit-sat-190923020628/85/Paper-study-Learning-to-solve-circuit-sat-21-320.jpg)
![Optimization
• 𝒮 𝜃: 𝒢 ↦ [0, 1] as 𝑆 𝜃 𝜇 𝐺 = ℛ 𝐺(ℱ 𝜃(𝜇 𝐺))
• Use the policy network to produce assignment and feed to reward
network to validate the satisfiability.
• Minimize the loss function
ℒ 𝑠 =
1−𝑠 𝜅
1−𝑠 𝜅+𝑠 𝜅 where 𝑠 = 𝒮 𝜃(𝜇 𝐺) and 𝜅 ≥ 1
• Could be seen as the portion of un-satisfiability.](https://image.slidesharecdn.com/learningtosolvecircuit-sat-190923020628/85/Paper-study-Learning-to-solve-circuit-sat-24-320.jpg)
![Graph k-coloring (cont’d)
• Given a graph with 𝑁 nodes and maximum 𝑘 allowed colors,
• Define 𝑥𝑖𝑗 for 1 ≤ 𝑖 ≤ 𝑁 and 1 ≤ 𝑗 ≤ 𝑘, where 𝑥𝑖𝑗 = 1 indicates
that the 𝑖th node is colored by the 𝑗th color.
• Transform the problem to SAT problem with Boolean CNF:
• ⋀𝑖=1
𝑁
(⋁𝑗=1
𝑘
𝑥𝑖𝑗) ∧ [⋀ 𝑝,𝑞 ∈𝐸(⋀𝑗=1
𝑘
(¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗))]
• Left set of clauses (red): ensure that each node of the graph takes at least
one color.
• Right set of clauses (blue): constrain that the neighboring nodes cannot
take the same color. 𝒙 𝒑𝒋 𝒙 𝒒𝒋 (¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗)
0 0 1
0 1 1
1 0 1
1 1 0](https://image.slidesharecdn.com/learningtosolvecircuit-sat-190923020628/85/Paper-study-Learning-to-solve-circuit-sat-29-320.jpg)
![Example
blue green red
1 T F F
2 F F T
3 F T F
4 F F T
1
3
2
4
⋀𝑖=1
𝑁
(⋁𝑗=1
𝑘
𝑥𝑖𝑗) ∧ [⋀ 𝑝,𝑞 ∈𝐸(⋀𝑗=1
𝑘
(¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗))]
colornode](https://image.slidesharecdn.com/learningtosolvecircuit-sat-190923020628/85/Paper-study-Learning-to-solve-circuit-sat-30-320.jpg)
Paper study: Learning to solve circuit sat
- 1. Learning to Solve Circuit-SAT: an Unsupervised Differentiable Approach Saeed Amizadeh, Sergiy Matusevych and Markus Weimer Microsoft ICLR 2019
- 2. Outline • Abstract • Introduction • Background • DAG embedding • Application to the Circuit-SAT problem • Experimental evaluation • Discussion
- 3. Abstract • New trend of applying rich neural architectures to solve classical combinatorial optimization problems. • Propose a neural framework that can learn to solve the Circuit-SAT problem. • Rich embedding architecture that encode the problem structure • Differentiable training procedure that mimics RL • Train the model directly toward solving the SAT problem • Show the superior out-of-sample generalization performance compared to NeuroSAT method.
- 4. Introduction
- 5. • The emergence of neural models for learning how to solve the classical combinatorial optimization problems. • In practice, for a given class of combinatorial problems, the problem instances are typically drawn from a certain (unknown) distribution. • If there are sufficient number of problem instances, ML (DL) is capable of extracting the common structures among these instances and produce models that would outperform the carefully hand-crafted algorithms.
- 6. • Supervised learning • Be shown to be effective for various NP-complete problems. • The resulted model is bounded by greedy strategy (sub-optimal in general). • Reinforcement learning • Learn the optimal decision and search heuristics beyond the greedy strategy. • Challenging work to train such model. • Propose a neural Circuit-SAT training the model directly toward the end goal.
- 7. Represent the instance • Classical architectures like RNNs or LSTMs • Ignore the inherent structure present in the problem instances. • Neural graph embedding embraces the information on graph that represents the problem structure. • Most methods synchronously propagate local information on an underlying graph. • The information is propagated sequentially rather than synchronously in the proposed model.
- 8. Background
- 10. SAT • The problem of determining if there exists an assignment that satisfies a given Boolean formula (consisted of variable, AND, OR and NOT). • E.g. • 𝑢1 ∨ ¬𝑢2 ∧ (¬𝑢1 ∨ 𝑢2) • 𝑢1 = 𝑢2 = TRUE → SAT • Variable: 𝑢1, 𝑢2 • Literals: 𝑢1, ¬𝑢1, 𝑢2, ¬𝑢2 • Clauses: 𝑢1 ∨ ¬𝑢2 , (¬𝑢1 ∨ 𝑢2) • Conjunctive normal form (CNF) • Conjunction (AND) of one or more clauses, where a clause is a disjunction (OR) of literals. Source: https://en.wikipedia.org/wiki/Conjunctive_normal_form
- 11. Circuit-SAT • The decision problem of determining whether a given Boolean circuit (only composed of AND, OR and NOT gate) has an assignment of its inputs that makes the output true. Source : https://en.wikipedia.org/wiki/Circuit_satisfiability_problem 𝒙 𝟏 𝒙 𝟐 𝒚 0 0 0 0 1 0 1 0 0 1 1 1 𝒙 𝒚 0 0 1 0 𝑥1 𝑥2 𝑦 𝑦𝑥 SAT UNSAT
- 12. NeuroSAT (Selsam et al. (2018)) • Approach the SAT problem as a binary classification problem. • Find the SAT solution by clustering the latent representation extracted from the learned classifier. • NOT directly trained toward finding SAT solutions. Source: https://github.com/dselsam/neurosat
- 13. Direct Acyclic Graph (DAG) • Directed graph with no directed cycles. • Topological sort • Given a directed graph 𝐺, a linear ordering of its vertices such that for every directed edge (𝑢, 𝑣), 𝑢 comes before 𝑣 in the ordering. Source: http://www.csie.ntnu.edu.tw/~u91029/DirectedAcyclicGraph.html
- 14. DAG embedding
- 15. • DAG 𝐺 = 𝑉𝐺, 𝐸 𝐺 • Reversed DAG 𝐺 𝑟 with the same set of nodes but reversed edges • For 𝑣 ∈ 𝑉𝐺, 𝜋 𝐺(𝑣) represents the set of direct predecessors • DAG function 𝜇 𝐺: 𝑉𝐺 ↦ ℝ 𝑑 • Define 𝒢 𝑑 as the space of all possible 𝑑-dimensional functions 𝜇 𝐺 • Model ℱ 𝜃 𝜇 𝐺 = 𝒞 𝒶(𝒫(ℰℬ(𝜇 𝐺))) • Embedding function ℰℬ: 𝒢 𝑑 ↦ 𝒢 𝑞 • Pooling function 𝒫: 𝒢 𝑞 ↦ 𝒢 𝑞 • Retrieve only the sink nodes in the input DAG • Classification function 𝒞 𝒶: 𝒢 𝑞 ↦ 𝒪
- 16. • For simplicity, define • Feature vector 𝒙 𝑣 = 𝜇 𝐺 𝑣 ∈ ℝ 𝑑 • State vector 𝒉 𝑣 = 𝛿 𝐺 𝑣 ∈ ℝ 𝑞 where 𝛿 𝐺: 𝑉𝐺 ↦ ℝ 𝑞 • Update logic that applying on each node: 𝒉 𝑣 = 𝐺𝑅𝑈 𝒙 𝑣, 𝒉 𝑣 ′ , where 𝒉 𝑣 ′ = 𝒜({𝒉 𝑢|𝑢 ∈ 𝜋(𝑣)}) • Aggregator function 𝒜: 2 𝑉 𝐺 ↦ ℝ 𝑞 • Aggregate state of node’s direct predecessors. • Apply the update logic sequentially in the topological sort order.
- 17. Deep-Gated DAG Recursive Neural Network (DG-DAGRNN) • Stack 𝐿-layer embedding layer where the 𝑖th layer has its own parameters ℬ𝑖 and output DAG function dimensionality 𝑞𝑖. • Apply sequentially 𝑇 times on stacked 𝐿 layers. ℰℬ 𝜇 𝐺 ≡ ℰℬ 𝑇 𝜇 𝐺 , where ℰℬ 𝑡 𝜇 𝐺 = ℰ 𝑠𝑡𝑎𝑐𝑘 𝑃𝑟𝑜𝑗 𝑯 ℰℬ 𝑡−1 𝜇 𝐺 , ∀𝑡 ∈ 2. . 𝑇 ℰℬ 1 𝜇 𝐺 = ℰ 𝑠𝑡𝑎𝑐𝑘(𝜇 𝐺) s. t. ℰ 𝑠𝑡𝑎𝑐𝑘 = ℰℬ 𝐿 ∘ ℰℬ 𝐿−1 ∘ ℰ1 • 𝑤ℎ𝑒𝑟𝑒 ℬ = 〈ℬ1, … , ℬ 𝐿, 𝑯〉 is the list of parameters and 𝑃𝑟𝑜𝑗 𝑯: 𝒢 𝑞 𝐿 ↦ 𝒢 𝑑 is the linear projection with the projection matrix 𝑯 𝑑×𝑞 𝐿 .
- 20. • DAG function 𝜇 𝐺: 𝑉𝐺 ↦ ℝ4 • 𝜇 𝐺 𝑣 = One−Hot 𝑡𝑦𝑝𝑒 𝑣 where 𝑡𝑦𝑝𝑒 v ∈ 𝐀𝐧𝐝, 𝐎𝐫, 𝐍𝐨𝐭, 𝐕𝐚𝐫𝐢𝐚𝐛𝐥𝐞 • Each node representing either a Boolean variable or a logical gate. 𝑥 1 2 3 0 0 0 1 0 0 1 0 1 0 0 0 Variables in sources Only one sink
- 21. Solver Network • Embedding function ℰℬ: • multi-layer recursive embedding with interleaving forward and reversed layers • Last layer is a reversed layer so we can read the output from Variable nodes. • Output space encode the soft assignment (in range [0−1]) to the corresponding variable node in the input circuit. • ℱ 𝜃 acts as policy network. Pooling: Retrieve only the sink nodes
- 22. Evaluator Network • 𝑆 𝑚𝑎𝑥 𝑎1, 𝑎2, … , 𝑎 𝑛 = σ 𝑖=1 𝑛 𝑎 𝑖 𝑒 Τ𝑎 𝑖 𝜏 σ𝑖=1 𝑛 𝑒 Τ𝑎 𝑖 𝜏 , 𝑆 𝑚𝑖𝑛 𝑎1, 𝑎2, … , 𝑎 𝑛 = σ 𝑖=1 𝑛 𝑎 𝑖 𝑒 Τ−𝑎 𝑖 𝜏 σ𝑖=1 𝑛 𝑒− Τ𝑎 𝑖 𝜏 • For 𝜏 = +∞, 𝑆 𝑚𝑎𝑥() and 𝑆 𝑚𝑖𝑛() are arithmetic mean. • E.g. • 𝒂 = 1, 2, 3 , 𝜏 = +∞ • 𝑆 𝑚𝑎𝑥 𝒂 = 𝑆 𝑚𝑖𝑛 𝒂 = 1⋅𝑒0+2⋅𝑒0+3⋅𝑒0 𝑒0+𝑒0+𝑒0 = 1+2+3 3 = 2 • For 𝜏 → 0, 𝑆 𝑚𝑎𝑥 → max(), 𝑆 𝑚𝑖𝑛 → min() • E.g. • 𝒂 = 1, 2, 3 , 𝜏 = 0.01 • 𝑆 𝑚𝑎𝑥 𝒂 = 1⋅𝑒 Τ1 0.01+2⋅𝑒 Τ2 0.01+3⋅𝑒 Τ3 0.01 𝑒 Τ1 0.01+𝑒 Τ2 0.01+𝑒 Τ3 0.01 = 1𝑒100+2𝑒200+3𝑒300 𝑒100+𝑒200+𝑒300 ≈ 3𝑒300 𝑒300 = 3 • 𝑆 𝑚𝑖𝑛 𝒂 = 1⋅𝑒 Τ−1 0.01+2⋅𝑒 Τ−2 0.01+3⋅𝑒 Τ−3 0.01 𝑒 Τ−1 0.01+𝑒 Τ−2 0.01+𝑒 Τ−3 0.01 = 1𝑒−100+2𝑒−200+3𝑒−300 𝑒−100+𝑒−200+𝑒−300 ≈ 𝑒−100 𝑒−100 = 1
- 23. Evaluator Network (cont’d) • For any given circuit 𝜇 𝐺, define the soft evaluation function ℛ 𝐺 as a DAG that shares the same topology 𝐺 with the circuit 𝜇 𝐺. • Except that replace each • And node to smooth min (𝑆 𝑚𝑎𝑥) • Or node to smooth max (𝑆 𝑚𝑖𝑛) • Not nodes to 𝒩 𝑧 = 1 − 𝑧 • At a low enough temperature (𝜏), if for a given input assignment, ℛ 𝐺 yields a value strictly greater than 0.5, then that assignment can be seen as a satisfying solution for the circuit. • ℛ 𝐺 acts as reward network.
- 24. Optimization • 𝒮 𝜃: 𝒢 ↦ [0, 1] as 𝑆 𝜃 𝜇 𝐺 = ℛ 𝐺(ℱ 𝜃(𝜇 𝐺)) • Use the policy network to produce assignment and feed to reward network to validate the satisfiability. • Minimize the loss function ℒ 𝑠 = 1−𝑠 𝜅 1−𝑠 𝜅+𝑠 𝜅 where 𝑠 = 𝒮 𝜃(𝜇 𝐺) and 𝜅 ≥ 1 • Could be seen as the portion of un-satisfiability.
- 26. • Baseline: NeuroSAT • Assume input problems come in CNF. • Convert the input CNF into circuit before feeding to proposed model (DG-DAGRNN). • Use the same generation process in NeuroSAT to produce the random dataset. • 300K SAT and UNSAT pairs • Number of Boolean variables: 3 to 10 • Performance metric: the percentage of SAT problems in the test set that each model can actually find a SAT solution for.
- 27. number of recurrences 𝑇 (for embedding) increases In-Sample test Out-of-Sample test (Use 3-10 variables during training) number of variables in test set
- 28. Graph k-coloring • Given an undirected graph 𝐺 with 𝑘 color values, in graph k- coloring decision problem, seek to find a mapping from the graph nodes to the color set such that no adjacent nodes in the graph have the same color. Source: https://en.wikipedia.org/wiki/Graph_coloring
- 29. Graph k-coloring (cont’d) • Given a graph with 𝑁 nodes and maximum 𝑘 allowed colors, • Define 𝑥𝑖𝑗 for 1 ≤ 𝑖 ≤ 𝑁 and 1 ≤ 𝑗 ≤ 𝑘, where 𝑥𝑖𝑗 = 1 indicates that the 𝑖th node is colored by the 𝑗th color. • Transform the problem to SAT problem with Boolean CNF: • ⋀𝑖=1 𝑁 (⋁𝑗=1 𝑘 𝑥𝑖𝑗) ∧ [⋀ 𝑝,𝑞 ∈𝐸(⋀𝑗=1 𝑘 (¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗))] • Left set of clauses (red): ensure that each node of the graph takes at least one color. • Right set of clauses (blue): constrain that the neighboring nodes cannot take the same color. 𝒙 𝒑𝒋 𝒙 𝒒𝒋 (¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗) 0 0 1 0 1 1 1 0 1 1 1 0
- 30. Example blue green red 1 T F F 2 F F T 3 F T F 4 F F T 1 3 2 4 ⋀𝑖=1 𝑁 (⋁𝑗=1 𝑘 𝑥𝑖𝑗) ∧ [⋀ 𝑝,𝑞 ∈𝐸(⋀𝑗=1 𝑘 (¬𝑥 𝑝𝑗 ∨ ¬𝑥 𝑞𝑗))] colornode
- 31. • Dataset 1 • number of nodes: 6-10 • edge percentage: 37% • Generate random graph from 6 different distributions and then pair with random k color in the range of 2 ≤ 𝑘 ≤ 4. • Dataset 2 • Generate random trees with the same node number in dataset 1. • Add random edges to graph until it become UNSAT. • Remove the last added edge to form SAT problem. • Proposed model could solve 48% and 27% in Dataset 1 and Dataset 2, respectively. • Surprisingly, NeuroSAT could not solve any problems, which does not consist to original paper. • NeuroSAT may be sensitive to the change of problem distribution.
- 32. Discussion • Propose a neural framework for efficiently learning a Circuit-SAT solver. • Rely on DAG-embedding architecture and efficient training procedure. • Solve the SAT problem in a fewer number of iterations compared to the baseline.