A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR WITH SMALLER FACTORS

International Journal of Soft Computing, Mathematics and Control (IJSCMC)
Vol 11, No 1/2/3/4, November 2022
1
A PROBABILISTIC ALGORITHM OF COMPUTING
THE POLYNOMIAL GREATEST COMMON DIVISOR
WITH SMALLER FACTORS
YangZhang1,2
, Xin Qian1,2
,Qidi You1,2
, Xuan Zhou1,2
,
Xiyong Zhang1,2
and Yang Wang1,2
1
Space star technology co., LTD
2
State Key Laboratory of Space-Ground Integrated Information Technology
ABSTRACT
In the earlier work, subresultant algorithm was proposed to decrease the coefficient growth in the
Euclidean algorithm of polynomials. However, the output polynomial remainders may have a small factor
which can be removed to satisfy our needs. Then later, an improved subresultant algorithm was given by
representing the subresultant algorithm in another way, where we add a variant called 𝜏 to express the
small factor. There was a way to compute the variant proposed by Brown, who worked at IBM.
Nevertheless, the way failed to determine each𝜏 correctly.
In this paper, we will give a probabilistic algorithm to determine the variant 𝜏 correctly in most cases by
adding a few steps instead of computing 𝑡 𝑥 when given 𝑓 𝑥 and𝑔 𝑥 ∈ ℤ 𝑥 , where 𝑡 𝑥 satisfies that
𝑠 𝑥 𝑓 𝑥 + 𝑡 𝑥 𝑔 𝑥 = 𝑟 𝑥 , here 𝑡 𝑥 , 𝑠 𝑥 ∈ ℤ 𝑥 .Also , we made experiments on the correctness and
efficiency of our algorithm, which has obvious advantage compared with the previous fastest algorithm.
KEYWORDS
Euclidean Algorithm, extended Euclidean Algorithm, Subresultant Algorithm, Primitive Remainder
Sequence, Ideal Lattice.
1. INTRODUCTION
The Euclidean algorithm and the extended Euclidean algorithm of polynomials are important
research objects in polynomial computer algebra. Using these algorithms, one can get the greatest
common divisor (short for G.C.D.) of two polynomials(denoted as gcd 𝑓, 𝑔 when given
polynomials 𝑓 𝑥 and 𝑔 𝑥 ) and decide whether these polynomials are coprime or not.
Specifically, if the degree of gcd 𝑓, 𝑔 is larger than 0, 𝑓 𝑥 and 𝑔 𝑥 are not coprime,
otherwise, 𝑓 𝑥 and 𝑔 𝑥 are coprime. Being coprime between two polynomials means that
there exists a common root between these two polynomials.
To quantify the indicator whether there exists a common root between 𝑓 𝑥 and 𝑔 𝑥 , Sylvester
gave a matrix in 1840 called Sylvester matrix with entries simply being the coefficients of 𝑓 𝑥
and 𝑔 𝑥 . The determinant of Sylvester matrix is called resultant. Whether the resultant of 𝑓 𝑥
and 𝑔 𝑥 is nonzero corresponds to the case where 𝑓 𝑥 and 𝑔 𝑥 are coprime or not
respectively. Moreover, Sylvester generalized his definition and introduced the concept of
subresultant. He mentioned that some subresultant is nonzero if and only if the corresponding
degree appears as a degree of a remainder of the Euclidean algorithm.

Vol 11, No 1/2/3/4, November 2022
2
However, the early Euclidean algorithm of polynomials works for polynomials in 𝔽 𝑥 , here 𝔽 is
a field. In 1836 Jacobi introduced pseudo-division over polynomials and extended the Euclidean
algorithm of polynomials in field to a ring by multiplying 𝑓 𝑥 with a certain power of the
leading coefficient of 𝑔 𝑥 before starting the division chain.
Using pseudo-division, there are a lot of results about polynomials even the ideal lattice used in
cryptography. From 1960, researchers built early computer algebra systems and G.C.D.
computations were an important test problem. Nevertheless, using pseudo-division in Euclidean
algorithm causes exponential coefficients growth, which cost huge storage and computing
resource. In 1967, Collins [1] explained that the 𝑖-th intermediate coefficients are approximately
longer by a factor of 1 + 2
𝑖
than the input coefficients. Although there are many ways to
decrease the size of coefficients during the Euclidean algorithm procedure, most of them are quite
inefficient. In this paper, we mainly focus on the subresultant algorithm and its variant. In [2],
Knuth present the early subresultant algorithm and gave an elegant proof of its correctness. The
subresultant decreases the large coefficients without computing lots of G.C.D.s during large
integers, which is the reason why the algorithm is expensive. In [3], Brown showed the variant of
subresultant algorithm and gave a way to remove the small factor of each remainder. However,
the method he present didn't work always.
Recently, in [4], they show an algorithm to triangularize the basis of an ideal lattice which is
often used to construct ideal lattice-based cryptosystems. They found the special relation between
the triangularization and the Euclidean algorithm of polynomials. In their algorithm, they need to
compute all the PPRSoL (the definition will be given in Sec.2.3.) of 𝑓 𝑥 and 𝑔 𝑥 . However,
to obtain the PPRSoL, we need to compute each content of 𝑡𝑖 𝑥 satisfying 𝑠𝑖 𝑥 𝑓 𝑥 +
𝑡𝑖 𝑥 𝑔 𝑥 = 𝑟𝑖 𝑥 to remove the extra factor in each original remaindes 𝑟𝑖 𝑥 . In this paper, we
find a new way to obtain PPRSoL without computing 𝑡𝑖 𝑥 by applying the variant of
subresultant algorithm, which is more efficient but probabilistic.
In this paper, we give some results about the extended Euclidean algorithm. Using these results,
we propose a new algorithm that outputting the PPRSoL of 𝑓 𝑥 and 𝑔 𝑥 which works for most
cases. In addition, we give the experimental results between the previous best algorithm with
ours.
Roadmap. In section 1, the motivation and contribution is presented. In section 2, we give the
preliminaries used in our algorithm. In section 3, some results of subresultant and the extended
Euclidean algorithm will be presented. In section 4, there shows our algorithm and experimental
results. A brief conclusion will be given at last.
2. PRELIMINARIES
2.1. Notations
In this paper, we use an uppercase bold letter to denote a matrix and a lowercase bold letter to
denote a vecto. The set of integers and the set of real number are denoted as ℤ and ℝ respectively.
For a matrix 𝑨 ∈ ℝ𝑚×𝑛
, the element in the 𝑖-th row and the 𝑗-th column position of 𝑨 is
expressed as 𝑎𝑖,𝑗 . For a polynomial 𝑓 𝑥 with degree 𝑛, we use 𝑙𝑐 𝑓 and 𝑑𝑒𝑔 𝑓 to present the
leading coefficient and the degree of 𝑓 𝑥 respectively. The degree of a nonzero constant
polynomial is defined as 0 and the degree of a zero polynomial is defined as −∞. The greatest
common divisor is abbreviated to G.C.D. and the G.C.D. of 𝑓 𝑥 and 𝑔 𝑥 is denoted as
gcd⁡
(𝑓, 𝑔). Let 𝜍 𝑥 denotethe ring of polynomials in 𝑥 with coefficients in 𝜍. Due to the

Vol 11, No 1/2/3/4, November 2022
3
application scenarios, unless otherwise specified, we only consider the polynomials in ℤ 𝑥 in
this paper.
2.2. Linear Algebra
Definition 1: [Lattice] Given 𝑚 linear independent vectors in ℝ𝑛
, 𝑏1, ⋯ , 𝑏𝑚 ∈ ℝ𝑛
, here 𝑚 ≤ 𝑛,
then the lattice ℒ generated by 𝑏1, ⋯ , 𝑏𝑚is defined as the integer coefficient linear combination
of 𝑏1, ⋯ , 𝑏𝑚, that is,
ℒ 𝑏1, ⋯ , 𝑏𝑚 = 𝑥𝑖𝑏𝑖
𝑚
𝑖=1
, 𝑥𝑖 ∈ ℤ.
Here 𝑏1, ⋯ , 𝑏𝑚form the basis of ℒ. If 𝑚 = 𝑛, the lattice ℒ is full-rank.
Definition 2: [Hermite Normal Form] Given a square matrix 𝑯 ∈ ℤ𝑛×𝑛
. Then 𝑯 is Hermite
Normal Form (HNF) if and only if it satisfies:
1) 𝑕𝑖,𝑖 ≥ 1, for 1 ≤ 𝑖 ≤ 𝑛;
2) 𝑕𝑖,𝑗 = 0, for 1 ≤ 𝑗 ≤ 𝑖 ≤ 𝑛;
3) 𝑕𝑗,𝑖 < 𝑕𝑖,𝑖, for 1 ≤ 𝑗 < 𝑖 ≤ 𝑛.
This is a very important structure in lattice-base cryptography. We usually regard the HNF of the
basis as the public key, because all the bases of a certain lattice have the same HNF which can be
computed in polynomial time. Thus using the HNF as public key ensures minimum leakage of
lattice basis.
We need to emphasize that the definition given above is only one way to define HNF. According
to row or column transformation and upper or lower triangularization, there are other different
definitions of HNF. In any event, HNF is a triangular matrix with certain divisibility relation.
2.3. Polynomial Theory
Definition 3: [Primitive Polynomial] A polynomial 𝑎 𝑥 ∈ ℤ 𝑥 is called a primitive polynomial
if for any integer 𝑘 > 1, 𝑎 𝑥 /𝑘 ∉ ℤ 𝑥 .
Definition 4: [Content] For a polynomial 𝑎 𝑥 = 𝑎𝑛 𝑥𝑛
+ ⋯ + 𝑎1𝑥 + 𝑎0 ∈ ℤ 𝑥 , the content of
𝑎 𝑥 , denoted as𝑐𝑜𝑛𝑡 𝑎 𝑥 , is the g.c.d of 𝑎𝑛 , ⋯ , 𝑎1, 𝑎0 .
Definition 5: [Resultant] Let 𝑓 𝑥 = 𝑓𝑛𝑥𝑛
+ ⋯ + 𝑓1𝑥 + 𝑓0, 𝑔 𝑥 = 𝑔𝑚 𝑥𝑚
+ ⋯ + 𝑔1𝑥 + 𝑔0 be
two polynomials with degree 𝑛 and 𝑚 respectively. Define the Sylvester matrix of 𝑓 𝑥 and 𝑔 𝑥
as

Vol 11, No 1/2/3/4, November 2022
4
𝐒𝐲𝐥𝐯 𝑓, 𝑔 =
𝑥𝑚−1
𝑓 𝑥
𝑥𝑚−2
𝑓 𝑥
⋮
𝑓 𝑥
𝑥𝑛−1
𝑔 𝑥
𝑥𝑛−2
𝑔 𝑥
⋮
𝑔 𝑥
=
𝑓𝑛 𝑓𝑛−1 ⋯ 𝑓0
⋱ ⋱
𝑔𝑚 𝑔𝑚−1 ⋯ 𝑔1 𝑔0
𝑔𝑚 𝑔𝑚−1 ⋯ 𝑔1 𝑔0
⋱ ⋱
𝑔𝑚 𝑔𝑚−1 ⋯ 𝑔1 𝑔0 𝑚+𝑛 × 𝑚+𝑛
Then the resultant of 𝑓 𝑥 and 𝑔 𝑥 , denoted as Res 𝑓 𝑥 , 𝑔 𝑥 , is the determinant of
𝐒𝐲𝐥𝐯 𝑓, 𝑔 .
Definition 6: [Subresultant] Let 𝑓 𝑥 = 𝑓𝑛𝑥𝑛
+ ⋯ + 𝑓1𝑥 + 𝑓0, 𝑔 𝑥 = 𝑔𝑚 𝑥𝑚
+ ⋯ + 𝑔1𝑥 + 𝑔0
be two polynomials with degree 𝑛 and𝑚 respectively. For 0 ≤ 𝑘 < 𝑛, the 𝑘-th subresultant of
𝑓 𝑥 and 𝑔 𝑥 is the determinant of 𝑆𝑘 𝑓, 𝑔 defines as
𝑆𝑘 𝑓, 𝑔 =
𝑓𝑛 𝑓𝑛−1 ⋯ 𝑓𝑛−𝑚+𝑘+1 ⋯ 𝑓𝑘+1 ⋯ 𝑓2𝑘−𝑚+1
𝑓𝑛 ⋯ 𝑓𝑛−𝑚+𝑘+2 ⋯ 𝑓𝑘+2 ⋯ 𝑓2𝑘−𝑚+2
⋱ ⋮ ⋮ ⋮
𝑓𝑛 ⋯ 𝑓𝑚 ⋯ 𝑓𝑘
𝑔𝑚 𝑔𝑚−1 ⋯ 𝑔𝑘+1 ⋯ 𝑔𝑚−𝑛+𝑘+1 ⋯ 𝑔2𝑘−𝑛+1
𝑔𝑚 ⋮ ⋮ ⋮
⋱ ⋮ ⋮ ⋮
𝑔𝑚 ⋮ ⋮
⋱ ⋮ ⋮
𝑔𝑚 ⋯ 𝑔𝑘 𝑚+𝑛−2𝑘 × 𝑚+𝑛−2𝑘
Remark 1. From the structure of 𝑆𝑘 𝑓, 𝑔 and 𝐒𝐲𝐥𝐯 𝑓, 𝑔 , we can tell that indeed if we delete the
last 2𝑘 columns and the last 𝑘 rows of 𝑓 𝑥 and 𝑔 𝑥 respectively in 𝐒𝐲𝐥𝐯 𝑓, 𝑔 , we obtain
𝑆𝑘 𝑓, 𝑔 . Expecially, 𝑆0 𝑓, 𝑔 = 𝐒𝐲𝐥𝐯 𝑓, 𝑔 .
Next we will give the conception of ideal lattice which takes an important role in the lattice-based
cryptography, and we mainly focus on the cases in which an ideal lattice can be derived from
𝑓 𝑥 and 𝑔 𝑥 .
Definition 7. [Ideal Lattice] We define ideal lattice over a ring 𝑅 = ℤ 𝑥 / 𝑓 𝑥 , where 𝑓 𝑥 ∈
ℤ 𝑥 is a monic and irreducible polynomial of degree 𝑛 and 𝑓 𝑥 is the ideal generated by
𝑓 𝑥 ∈ ℤ 𝑥 .
Consider the coefficient embedding
𝜎：𝑅 ↦ ℤ𝑛
𝑎𝑖𝑥𝑖
𝑛−1
𝑖=0
↦ 𝑎𝑛−1, 𝑎𝑛−2, ⋯ , 𝑎0
From [5], we know that the ideal generated by 𝑔 𝑥 forms a lattice under 𝜎 and we call it the
ideal lattice ℒ generated by 𝑔 𝑥 . Moreover, 𝑔 𝑥 mod𝑓 𝑥 , 𝑥𝑔 𝑥 mod𝑓 𝑥 , ⋯ ,
𝑥𝑛−1
𝑔 𝑥 mod𝑓 𝑥 form a basis of ℒ. As we can see, the basis is closely related to the Sylvester

Vol 11, No 1/2/3/4, November 2022
5
matrix of 𝑓 𝑥 and 𝑔 𝑥 . When 𝑓 𝑥 and 𝑔 𝑥 are coprime over ℚ 𝑥 , the ideal lattice is full-
rank.
From [4], they combined the extended Euclidean algorithm with the triangularization of the basis
over ideal lattice and found the relationship between the two algorithms. This is the reason why
we mainly focus on the extended Euclidean algorithm and try to improve the efficiency of the
procedure.
Then we present a lemma from [5] that will be used to prove our results later.
Lemma 1. Let ℒ be the ideal lattice generated by 𝑔 𝑥 ∈ 𝑅 = ℤ 𝑥 / 𝑓 𝑥 , where 𝑓 𝑥 is a
monic polynomial of degree 𝑛 and is relatively prime to 𝑔 𝑥 . Then the Hermite Normal Form of
a basis of ℒ
𝐻 =
𝑕1,1 𝑕1,2 ⋯ 𝑕1,𝑛
𝑕2,2 ⋯ 𝑕2,𝑛
⋱ ⋮
𝑕𝑛,𝑛
satisfies 𝑕𝑖,𝑖|𝑕𝑙,𝑗 , for 1 ≤ 𝑖 ≤ 𝑙 ≤ 𝑗 ≤ 𝑛.
2.3.1. Euclidean Algorithm of Polynomials over A Field
Given a field 𝔽. Let 𝑓 𝑥 and 𝑔 𝑥 ∈ 𝔽 𝑥 with 𝑑𝑒𝑔 𝑓 > 𝑑𝑒𝑔 𝑔 . Then the division of 𝑓 𝑥
and 𝑔 𝑥 yields a unique quotient 𝑄 𝑥 and remainder 𝑅 𝑥 such that
𝑓 𝑥 = 𝑄 𝑥 𝑔 𝑥 + 𝑅 𝑥
here 𝑑𝑒𝑔 𝑔 > 𝑑𝑒𝑔 𝑅 , 𝑑𝑒𝑔 𝑄 = 𝑑𝑒𝑔 𝑓 − 𝑑𝑒𝑔 𝑔 .
If we repeat the step for each divisor polynomial and remainder in a division procedure, we will
obtain a sequence of remainders with decreasing degrees. Formally, a detailed procedure of the
Euclidean algorithm of polynomials over a field is present as following:
𝑓 𝑥 = 𝑄1 𝑥 𝑔 𝑥 + 𝑅1 𝑥
𝑔 𝑥 = 𝑄2 𝑥 𝑅1 𝑥 + 𝑅2 𝑥
⋮
𝑅𝑙−2 𝑥 = 𝑄𝑙 𝑥 𝑅𝑙−1 𝑥 + 𝑅𝑙 𝑥
𝑅𝑙−1 𝑥 = 𝑄𝑙+1 𝑥 𝑅𝑙 𝑥
where 𝑑𝑒𝑔 𝑔 > 𝑑𝑒𝑔 𝑅1 > ⋯ > 𝑑𝑒𝑔 𝑅𝑙 and all the coefficients are in the given field. Note
that if deg 𝑅𝑙 = 0, it shows that 𝑓 𝑥 and 𝑔 𝑥 are coprime in 𝔽 𝑥 , which means the resultant
of 𝑓 𝑥 and 𝑔 𝑥 is nonzero.
2.3.2. Polynomial Remainder Sequence
The procedure of the Euclidean algorithm of polynomials over a unique factorization domain
(UFD) is similar to the one over a field. The difference exits because the division between two
polynomials requires exact divisibility relation in the given ring, which is usually impossible to
realize. To solve the problem, the procedure of pseudo-division is proposed, which yields a
unique pseudo-quotient 𝑞 𝑥 and pseudo-remainder 𝑟 𝑥 such that

Vol 11, No 1/2/3/4, November 2022
6
lc 𝑔
𝛿+1
𝑓 𝑥 = 𝑞 𝑥 𝑔 𝑥 + 𝑟 𝑥
here 𝑑𝑒𝑔 𝑔 > 𝑑𝑒𝑔 𝑟 , 𝛿 = 𝑑𝑒𝑔 𝑓 − 𝑑𝑒𝑔 𝑔 , 𝑟 𝑥 is denoted as prem 𝑓, 𝑔 . Moreover, the
coefficients of 𝑞 𝑥 and 𝑟 𝑥 are in the given ring.
For nonzero polynomials 𝑎 𝑥 , 𝑏 𝑥 ∈ 𝜍 𝑥 , we say 𝑎 𝑥 is similar to𝑏 𝑥 (𝑎 𝑥 ~𝑏 𝑥 ) if there
exist 𝑐1, 𝑐2 ∈ 𝜍 such that 𝑐1𝑎 𝑥 = 𝑐2𝑏 𝑥 . So if we choose 𝑟′
𝑥 that is similar to 𝑟 𝑥 , we can
do the same step as above for 𝑔 𝑥 and 𝑟′
𝑥 . Thus, we can rewrite the procedure of pseudo-
division:
𝛼𝑓 𝑥 = 𝑞 𝑥 𝑔 𝑥 + 𝛽𝑟 𝑥 .
Then the detailed procedure of pseudo-division is present as following:
𝛼1𝑓 𝑥 = 𝑞1 𝑥 𝑔 𝑥 + 𝛽1𝑟1 𝑥
𝛼2𝑔 𝑥 = 𝑞2 𝑥 𝑟1 𝑥 + 𝛽2𝑟2 𝑥
⋮
𝛼𝑙𝑟𝑙−2 𝑥 = 𝑞𝑙 𝑥 𝑟𝑙−1 𝑥 + 𝛽𝑙𝑟𝑙 𝑥
𝛼𝑙+1𝑟𝑙−1 𝑥 = 𝑞𝑙+1 𝑥 𝑟𝑙 𝑥
here 𝑑𝑒𝑔 𝑔 > 𝑑𝑒𝑔 𝑟1 > ⋯ > 𝑑𝑒𝑔 𝑟𝑙 and all the 𝛼𝑖 and 𝛽𝑖 are in the given ring.
Generally, we denote 𝑓 𝑥 = 𝑟−1(𝑥) and 𝑔 𝑥 = 𝑟0 𝑥 , then 𝛼𝑖 = 𝑙𝑐 r𝑖−1
𝛿𝑖−2−1
, where
𝛿𝑖 = 𝑑𝑒𝑔 𝑟𝑖 − 𝑑𝑒𝑔 𝑟𝑖+1 . Note that prem 𝑟𝑖−2, 𝑟𝑖−1 = 𝛽𝑖𝑟𝑖(𝑥). Then 𝑟−1 𝑥 , 𝑟0 𝑥 ,⋯,𝑟𝑙 𝑥
form a sequence called polynomial remainder sequence (PRS).
From [4], if a remainder 𝑟 𝑥 = 𝑠 𝑥 𝑓 𝑥 + 𝑡 𝑥 𝑔 𝑥 can derive a basis of ideal lattice, 𝑡 𝑥
must be primitive. In this paper, we also want to obtain such remainders and we call these
remainders as primitive PRS of lattice (PPRSoL). Next, we give a result about 𝑠𝑖 𝑥 and 𝑡𝑖 𝑥 in
[7].
Lemma 2. Let 𝑓 𝑥 , 𝑔 𝑥 ∈ ℤ[𝑥] be two polynomials with degree 𝑛 and 𝑚 respectively, where
𝑛 > 𝑚 . Let 𝑟−1 𝑥 , 𝑟0 𝑥 , ⋯ , 𝑟𝑙 𝑥 be the remainders in procedure of pseudo-division. If
𝑑𝑒𝑔 𝑟𝑖 = 𝑛𝑖, then 𝑟𝑖 𝑥 = 𝑠𝑖 𝑥 𝑓 𝑥 + 𝑡𝑖 𝑥 𝑔 𝑥 satisfies 𝑑𝑒𝑔 𝑠𝑖 < 𝑚, 𝑑𝑒𝑔 𝑡𝑖 < 𝑛 and:
1) 𝑠𝑖 𝑥 = 𝛼𝑖𝑠𝑖−2 𝑥 − 𝑞𝑖 𝑥 𝑠𝑖−1 𝑥 /𝛽𝑖, 𝑡𝑖 𝑥 = 𝛼𝑖𝑡𝑖−2 𝑥 − 𝑞𝑖 𝑥 𝑡𝑖−1 𝑥 /𝛽𝑖
2) 𝑑𝑒𝑔 𝑠𝑖 = 𝑚 − 𝑑𝑒𝑔 𝑟𝑖−1 ,𝑑𝑒𝑔 𝑡𝑖 = 𝑛 − 𝑑𝑒𝑔 𝑟𝑖−1
If we represent 𝑟𝑖 𝑥 = 𝑠𝑖 𝑥 𝑓 𝑥 + 𝑡𝑖 𝑥 𝑔 𝑥 under the embedding σ, for 𝑖 = −1, 0, ⋯ , 𝑙, then
we can denote 𝑟𝑖 𝑥 as a sequence of vectors and we use a matrix 𝑹 to represent 𝑟𝑖 𝑥 as
following:

Vol 11, No 1/2/3/4, November 2022
7
𝑓𝑛 ⋯ 𝑓𝑛−𝑚+1 𝑓𝑛−𝑚 ⋯ 𝑓0
⋱ ⋱
𝑓𝑛 𝑓𝑛−1 ⋯ 𝑓𝑚−1 𝑓𝑚−2 ⋯ 𝑓1 𝑓0
𝑟0,𝑛0
⋯ 𝑟0,0
⋱
⋮ 𝑟0,𝑛0
⋯ 𝑟0,0
𝑟1,𝑛0
⋯ 𝑟1,0
𝟎 ⋱
𝑟1,𝑛1
⋯ 𝑟1,0
⋱
⋮ 𝑟𝑙,𝑛𝑙
⋱
𝑟𝑙,𝑛𝑙 𝑚+𝑛 × 𝑚+𝑛
Also, we use 𝑺 and 𝑻 to represent the matrice denoting 𝑠𝑖 𝑥 and 𝑡𝑖 𝑥 respectively.
𝑻 =
1
⋱
1
𝑡1,𝑛−𝑛0
⋯ 𝑡1,0
⋱ ⋱
𝑡1,𝑛−𝑛0
⋯ 𝑡1,0
⋱
𝑡𝑙,𝑛−𝑛𝑙−1
⋯ 𝑡𝑙,0
⋱ ⋱
𝑡𝑙,𝑛−𝑛𝑙−1
⋯ 𝑡𝑙,0 𝑛×𝑛
𝑺 =
0
⋱
0
𝑠1,𝑚−𝑛0
⋯ 𝑠1,0
⋱ ⋱
𝑠1,𝑚−𝑛0
⋯ 𝑠1,0
⋱
𝑠𝑙,𝑚−𝑛𝑙−1
⋯ 𝑠𝑙,0
⋱ ⋱
𝑠𝑙,𝑚−𝑛𝑙−1
⋯ 𝑠𝑙,0 𝑛×𝑚
So if we give a matrix named 𝑺𝑻, then procedure of pseudo-division can be represented as a
matrix multiplication.
𝑺𝑻 =
𝑰𝑛×𝑛 𝟎
𝑺 𝑻 𝑚+𝑛 × 𝑚+𝑛
=
1
⋱ 𝟎
1
𝑺 𝑻 𝑚+𝑛 × 𝑚+𝑛
Here 𝑰𝑛×𝑛 means the 𝑛 × 𝑛 identity matrix. Then

Vol 11, No 1/2/3/4, November 2022
8
𝑺𝑻 ∙ 𝐒𝐲𝐥𝐯 𝑓, 𝑔 = 𝑹
Here we need to show that by elementary row transformation, 𝑺𝑻 can be transformed into
𝑺𝑻 =
1
⋱ 𝟎
1
𝟎 𝑻 𝑚+𝑛 × 𝑚+𝑛
which means that the determinant of 𝑺𝑻 equals to the determinant of 𝑻. Also according to lemma
2, it turns out that after appropriate row transforming, 𝑻 is actually an upper triangular matrix,
thus the determinant of 𝑻 is lc 𝑡𝑖
𝑛𝑖−1−𝑛𝑖
𝑙
𝑖=0 .
In the following part, we introduce some typical PRSs which differs from each other by choosing
different 𝛽𝑖.
2.3.2.1. Euclidean Polynomial Remainder Sequences
When choosing 𝛽𝑖 = 1 for all 𝑖 in PRS, we obtain Euclidean PRS. This is a generalization of the
extended Euclidean algorithm over integers. However, the algorithm is quite inefficient because
with the proceeding of the algorithm, the coefficients of the divisor polynomials and remainders
grow exponentially. To be specific, we need to calculate each 𝑡𝑖 𝑥 and 𝑐𝑜𝑛𝑡 𝑡𝑖 𝑥 to get a
eligible PPRSoL, which costs too much. So we need to determine certain 𝛽𝑖 to ensure the
efficiency.
2.3.2.2. Primitive Polynomial Remainder Sequences
When choosing 𝛽𝑖 = 𝑐𝑜𝑛𝑡 prem 𝑟𝑖−2, 𝑟𝑖−1 for all 𝑖 in PRS, we obtain primitive PRS.
Although the algorithm stops the coefficients growing exponentially in every step of the pseudo-
division, however, during the procedure of obtaining primitive PRS, the coefficients of 𝑠𝑖 𝑥 and
𝑡𝑖 𝑥 are not in the given ring, which means that the PRS we obtain is not PPRSoL. So primitive
PRS doesn't satisfy our requirement.
2.3.2.3. Subresultant Polynomial Remainder Sequences
When 𝛽𝑖 is related to the subresultant, we obtain subresultant PRS. The equation set as following
depicts the procedure of the subresultant PRS algorithm in [2].
𝛼′1𝑓 𝑥 = 𝑞′1 𝑥 𝑔 𝑥 + 𝛽′1𝑟′1(𝑥)
𝛼′2𝑔 𝑥 = 𝑞′2 𝑥 𝑟1 𝑥 + 𝛽′2𝑟′2(𝑥)
⋮
𝛼′𝑙𝑟′𝑙−2 𝑥 = 𝑞′𝑙 𝑥 𝑟𝑙−1 𝑥 + 𝛽′𝑙𝑟′𝑙(𝑥)
where 𝑟−1 𝑥 = 𝑓(𝑥), 𝑟0 𝑥 = 𝑔(𝑥), 𝑛𝑖 = 𝑑𝑒𝑔 𝑟′𝑖 , 𝛿𝑖 = 𝑛𝑖 − 𝑛𝑖+1 , 𝛼′𝑖 = lc 𝑟′𝑖−1
𝛿𝑖−2+1
,
𝛽′𝑖 = lc 𝑟′𝑖−2 𝑕𝑖
𝛿𝑖−2
, 𝑕1 = 1, 𝑕𝑖 = (𝑙𝑐 𝑟′𝑖−2 )𝛿𝑖−3 𝑕𝑖−1
1−𝛿𝑖−3
, for 2 ≤ 𝑖 ≤ 𝑙 + 1.
Intuitively, the intact subresultant algorithm can be present in Algorithm 1. We point out that
because we want to get PPRSoL, the input of every PRS algorithm in the paper contains a monic
and irreducible polynomial.

Vol 11, No 1/2/3/4, November 2022
9
Algorithm 1 Subresultant PRS Algorithm
Input: two polynomials 𝑓 𝑥 , 𝑔 𝑥 ∈ ℤ[𝑥] with degree 𝑛 and 𝑚 respectively and 𝑓 𝑥 is monic and
irreducible
Output: Subresultant PRS, 𝑟′0 𝑥 , 𝑟′1 𝑥 , ⋯
1.[Initialize] 𝑙 ← 𝑕 ← 1, 𝑟′0 𝑥 = 𝑔 𝑥 ,𝑖 ← 1
2.[Pseudo-division]
2.1 Set δ = 𝑑𝑒𝑔 𝑓 − 𝑑𝑒𝑔 𝑔
2.2 Calculate 𝑟(𝑥) such that 𝑟(𝑥) = 𝑠(𝑥)𝑓(𝑥) + 𝑡(𝑥)𝑔(𝑥)
3.[Adjust remainder]
3.1 𝑢(𝑥) ← 𝑔(𝑥), 𝑟′𝑖 𝑥 ← 𝑔(𝑥) ← 𝑟(𝑥)/𝑙𝑕𝛿
3.2 𝑙 ← 𝑙𝑐(𝑓),𝑕 ← 𝑕1−𝛿
𝑙𝛿
3.3 If 𝑑𝑒𝑔(𝑟) = 0, go to Step 4
3.4 𝑖 ← 𝑖 + 1, go to Step 2
4.[Return] 𝑟′0 𝑥 , 𝑟′1 𝑥 , ⋯
Notice that for 𝑟′𝑖 𝑥 = 𝑠′𝑖 𝑥 𝑓(𝑥) + 𝑡′𝑖 𝑥 𝑔(𝑥), 𝑡′𝑖 𝑥 maybe not primitive in the given ring,
which means that we can still decrease the coefficients of 𝑟′𝑖 𝑥 by removing a small factor.
In [6], the author shows that the 𝑕𝑖 is indeed the 𝑛𝑖−1-th subresultant of 𝑓 𝑥 and 𝑔 𝑥 , that is
𝑕𝑖 = 𝑆𝑛𝑖−1
𝑓, 𝑔 . Also, in [3], the author shows that every 𝑕𝑖 is an integer and 𝑟′𝑖 𝑥 ∈ ℤ[𝑥].
Moreover he gives an elegant proof of the correctness of the algorithm.
2.3.2.4. Improvements of Subresultant Polynomial Remainder Sequences
This is another expression of subresultant PRS. As stated above, for the output of Algorithm 1,
𝑟′𝑖 𝑥 = 𝑠′𝑖 𝑥 𝑓(𝑥) + 𝑡′𝑖 𝑥 𝑔(𝑥), 𝑡′𝑖 𝑥 maybe not primitive and there might exist a divisor 𝜏𝑖
such that 𝑡𝑖 𝑥 = 𝑡′𝑖 𝑥 /𝜏𝑖 is primitive. So in the improvement version, the author transforms the
procedure of the subresultant PRS algorithm as following,
𝛼1𝑓 𝑥 = 𝑞1 𝑥 𝑔 𝑥 + 𝛽1𝑟1(𝑥)
𝛼2𝑔 𝑥 = 𝑞2 𝑥 𝑟1 𝑥 + 𝛽2𝑟2(𝑥)
⋮
𝛼𝑙𝑟𝑙−2 𝑥 = 𝑞𝑙 𝑥 𝑟𝑙−1 𝑥 + 𝛽𝑙𝑟𝑙(𝑥)
where 𝑟−1 𝑥 = 𝑓(𝑥) , 𝑟0 𝑥 = 𝑔(𝑥) , 𝑕1 = 1 , 𝑛𝑖 = 𝑑𝑒𝑔 𝑟𝑖 , 𝛿𝑖 = 𝑛𝑖 − 𝑛𝑖+1 ,
𝛼𝑖 = 𝑙𝑐 𝑟𝑖−1
𝛿𝑖−2+1
, 𝛽𝑖 = 𝑙𝑐 𝑟𝑖−2 𝑕𝑖
𝛿𝑖−2
𝜏𝑖−1
−𝛿𝑖−2−1
𝜏𝑖 , 𝑕𝑖 = (𝜏𝑖−2𝑙𝑐 𝑟𝑖−2 )𝛿𝑖−3 𝑕𝑖−1
1−𝛿𝑖−3
, for
2 ≤ 𝑖 ≤ 𝑙 + 1. 𝜏𝑖 is an integer such that 𝑡′𝑖 𝑥 /𝜏𝑖is a primitive polynomial. Clearly, 𝜏0 = 1. In
[3], the author chose 𝜏𝑖 = 𝑙𝑐 𝑟𝑖−1 if 𝑙𝑐 𝑟𝑖−1 |𝑟′𝑖 𝑥 , otherwise 𝜏𝑖 = 1. However, the method to
choose 𝜏𝑖 doesn't work for each 𝜏𝑖.
Comparing the two kinds of subresultant algorithms, we need to emphasis that all the 𝑕𝑖s are
equal in the two algorithms.
3. SOME PROPERTIES OF THE SUBRESULTANT POLYNOMIAL REMAINDER
SEQUENCE
Before presenting our algorithm, we first give some results about the subresultant PRS and the
extend Euclidean algorithm.
Proposition 1. Given two polynomials 𝑎(𝑥) = 𝑎𝑛 𝑥𝑛
+ ⋯ + 𝑎1𝑥 + 𝑎0 and 𝑏(𝑥) = 𝑏𝑚 𝑥𝑚
+ ⋯ +
𝑏1𝑥 + 𝑏0 ∈ ℤ[𝑥], where 𝑛 > 𝑚. Write 𝑏𝑚
𝑛−𝑚+1
𝑎 𝑥 = 𝑞 𝑥 𝑏 𝑥 + 𝑟 𝑥 . Define the matrix

Vol 11, No 1/2/3/4, November 2022
10
𝑴 =
𝑎𝑛 𝑎𝑛−1 ⋯ 𝑎𝑛−𝑚+1 𝑎𝑛−𝑚 ⋯ 𝑎2 𝑎1 𝑎0
𝑏𝑚 𝑏𝑚−1 ⋯ 𝑏1 𝑏0
⋱
If the determinant of the matrix 𝑴𝑖 is denoted as ∆𝑖 , where 𝑴𝑖 is the 𝑖 × 𝑖 submatrix of 𝑴
obtained by deleting the last (𝑛 − 𝑚 + 2 − 𝑖) rows and the last (𝑛 + 1 − 𝑖) columns from 𝑴,
𝑖 = 0, … , 𝑛 − 𝑚 + 1. Then 𝑞(𝑥) = ∆𝑛−𝑚+1−𝑖𝑏𝑚
𝑖
𝑥𝑖
𝑛−𝑚
𝑖=0 . Moreover, we have the divisibility
relation among 𝑎 𝑥 , 𝑏 𝑥 , 𝑞(𝑥), that is 𝑐𝑜𝑛𝑡 𝑎 𝑥 𝑐𝑜𝑛𝑡 𝑏 𝑥
𝑛−𝑚
|𝑞(𝑥).
Proof. We first give the detail of the pseudo-division procedure,
𝑏𝑚 𝑎 𝑥 = 𝑎𝑛 𝑥𝑛−𝑚
𝑏 𝑥 + 𝑅1(𝑥)
𝑏𝑚 𝑅1 𝑥 = 𝑙𝑐(𝑅1)𝑥𝑛−𝑚−1
𝑏 𝑥 + 𝑅2(𝑥)
⋮
𝑏𝑚 𝑅𝑛−𝑚−1 𝑥 = 𝑙𝑐 𝑅𝑛−𝑚−1 𝑥𝑏 𝑥 + 𝑅𝑛−𝑚 𝑥
𝑏𝑚 𝑅𝑛−𝑚 𝑥 = 𝑙𝑐 𝑅𝑛−𝑚 𝑥𝑏 𝑥 + 𝑟 𝑥
We denote 𝑅0 𝑥 = 𝑎(𝑥) and 𝑅𝑛−𝑚+1 𝑥 = 𝑟(𝑥) , then we claim that 𝑅𝑖 𝑥 = ∆𝑖,𝑗 𝑥𝑗
𝑛−𝑖
𝑗=0 ,
where ∆𝑖,𝑗 is the determinant of the 𝑖 + 1 × 𝑖 + 1 matrix 𝑀𝑖,𝑗 obtained by deleting the last
(𝑛 − 𝑚 + 1 − 𝑖) rows and the last (𝑛 + 1 − 𝑖) columns except column (𝑛 + 1 − 𝑗) from 𝑀,
𝑖 = 0, … , 𝑛 − 𝑚 + 1, 𝑗 = 0, … , 𝑛 − 𝑖. Clearly, ∆𝑖+1= ∆𝑖,𝑛−𝑖.
Then we explain the claim by induction on 𝑖, 𝑖 = 0, … , 𝑛 − 𝑚 + 1.
For 𝑖 = 0, we have 𝑅0 𝑥 = 𝑎 𝑥 and it's obvious that 𝑎𝑗 = ∆0,𝑗 for 𝑗 = 0, … , 𝑛.
Next we assume that the claim holds for 𝑖 = 𝑘 − 1. Then we denote 𝑏𝑚 𝑅𝑘−1(𝑥) and 𝑏 𝑥 as
following,
𝑏𝑚 ∆𝑘−1,𝑛+1−𝑘 𝑏𝑚 ∆𝑘−1,𝑛−𝑘 ⋯ 𝑏𝑚 ∆𝑘−1,𝑛−𝑚+1−𝑘 ⋯ ⋯ 𝑏𝑚 ∆𝑘−1,1 𝑏𝑚 ∆𝑘−1,0
𝑏𝑚 𝑏𝑚−1 ⋯ 𝑏0 0 ⋯ 0 0
Then the coefficient of 𝑥𝑛−𝑘+1−𝑖
in 𝑅𝑘(𝑥) is 𝑏𝑚 ∆𝑘−1,𝑛+1−𝑘−𝑖 − 𝑏𝑚−𝑖∆𝑘−1,𝑛+1−𝑘 if 1 ≤ 𝑖 ≤ 𝑚
and 𝑏𝑚 ∆𝑘−1,𝑛+1−𝑘−𝑖 otherwise. According to the structure of 𝑀 we know that the coefficient of
𝑥𝑛−𝑘+1−𝑖
is exactly ∆𝑘,𝑛−𝑘+1−𝑖. So the claim holds.
From the claim we have 𝑙𝑐 𝑅𝑖 = ∆𝑖,𝑛−𝑖= ∆𝑖+1 , so 𝑞(𝑥) = 𝑙𝑐 𝑅𝑛−𝑚−𝑖 𝑏𝑚
𝑖
𝑥𝑖
𝑛−𝑚
𝑖=0 =
∆𝑛−𝑚+1−𝑖𝑏𝑚
𝑖
𝑥𝑖
𝑛−𝑚
𝑖=0 .
Then from the structure of 𝑀𝑖, we know 𝑐𝑜𝑛𝑡 𝑎 𝑥 𝑐𝑜𝑛𝑡 𝑏 𝑥
𝑛−𝑚−𝑖
|∆𝑛−𝑚+1−𝑖. So
𝑐𝑜𝑛𝑡 𝑎 𝑥 𝑐𝑜𝑛𝑡 𝑏 𝑥
𝑛−𝑚
|∆𝑛−𝑚+1−𝑖𝑏𝑚
𝑖
,
which means 𝑐𝑜𝑛𝑡 𝑎 𝑥 𝑐𝑜𝑛𝑡 𝑏 𝑥
𝑛−𝑚
|𝑞(𝑥).
•

Vol 11, No 1/2/3/4, November 2022
11
Proposition 2. Let 𝑟1(𝑥), ⋯ , 𝑟𝑙(𝑥) be the remainders obtained in improved subresultant
algorithm. Present 𝑟𝑖(𝑥) = 𝑠𝑖(𝑥)𝑓(𝑥) + 𝑡𝑖(𝑥)𝑔(𝑥), for 𝑖 = 1, ⋯ , 𝑙. Then we have 𝑙𝑐 𝑡𝑖 =
𝑕𝑖+1
𝜏𝑖
.
Proof. According to Lemma 2, 𝑡𝑖 𝑥 =
1
𝛽𝑖
(𝛼𝑖𝑡𝑖−2 𝑥 − 𝑞𝑖 𝑥 𝑡𝑖−1 𝑥 ) and 𝑑𝑒𝑔 𝑡𝑖 = 𝑛 − 𝑛𝑖−1.
Also 𝑑𝑒𝑔 𝑞𝑖 = 𝛿𝑖−2 , so 𝑑𝑒𝑔 𝑡𝑖 = 𝑛 − 𝑛𝑖−3 < 𝑑𝑒𝑔 𝑞𝑖𝑡𝑖−1 = 𝑛 − 𝑛𝑖−1 . Then 𝑙𝑐 𝑡𝑖 =
1
𝛽𝑖
𝑙𝑐(𝑞𝑖) 𝑙𝑐 𝑡𝑖−1 , so 𝑙𝑐 𝑞𝑖 =
𝑙𝑐 𝑟𝑖−2
𝛼𝑖. Then 𝑙𝑐 𝑡𝑖 =
𝑙𝑐 𝑟𝑖−2 𝛼𝑖
𝑙𝑐 𝑟𝑖−1 𝛽𝑖
𝑙𝑐 𝑡𝑖−1 =
𝛼1…𝛼𝑖
𝛽1…𝛽𝑖𝑙𝑐 𝑟𝑖−1
.
Because 𝛼𝑖 = 𝑙𝑐 𝑟𝑖−1
𝛿𝑖−2+1
, 𝛽𝑖 = 𝑙𝑐 𝑟𝑖−1 𝑕𝑖
𝛿𝑖−2
𝜏𝑖−1
−𝛿𝑖−2−1
𝜏𝑖, then we have
𝑙𝑐 𝑡𝑖 =
1
(𝜏𝑖−1 𝑙𝑐 𝑟𝑖−1 )𝛿𝑖−2+1
𝑙𝑐 𝑟𝑖−2 𝑕𝑖
𝛿𝑖−2
𝜏𝑖
(𝜏𝑖−2 𝑙𝑐 𝑟𝑖−2 )𝛿𝑖−3+1
𝑙𝑐 𝑟𝑖−3 𝑕𝑖−1
𝛿𝑖−3
𝜏𝑖−1
…
(𝜏0 𝑙𝑐 𝑟0 )𝛿−1+1
𝑙𝑐 𝑟−1 𝑕1
𝛿−1
𝜏1
=
1
𝜏𝑖
(𝜏𝑖−1 𝑙𝑐 𝑟𝑖−1 )𝛿𝑖−2
𝑕𝑖
𝛿𝑖−2
𝑕𝑖−1
𝛿𝑖−3
…
(𝜏0 𝑙𝑐 𝑟0 )𝛿−1
𝑕1
𝛿−1
=
1
𝜏𝑖
𝑕𝑖
𝛿𝑖−2
𝑕𝑖−1
𝛿𝑖−3
…
(𝜏0 𝑙𝑐 𝑟1 )𝛿0
𝑕1
𝛿−1
𝑕2
= ⋯ =
𝑕𝑖+1
𝜏𝑖
•
Remark 2. If we do similar steps for 𝑟′0 𝑥 , 𝑟′1 𝑥 , ⋯ , 𝑟′𝑙 𝑥 in Algorithm 1 and present each
𝑟′𝑖 𝑥 = 𝑠′𝑖 𝑥 𝑓(𝑥) + 𝑡′𝑖 𝑥 𝑔(𝑥), then we obtain 𝑙𝑐 𝑡′𝑖 = 𝑕𝑖+1.
Before giving next lemmas, we first present a useful algorithm from [5]. We use the same
symbols in [5].{𝑛 − 𝑛𝑖−1 + 1, ⋯ , 𝑛 − 𝑛𝑖} = 𝐼𝑖, then {1,2, ⋯ , 𝑛} = 𝐼𝑖
𝑙
𝑖+1 .
Algorithm 2 A Useful Algorithm
Input:𝑟0 𝑥 , 𝑟1 𝑥 , ⋯ , 𝑟𝑙 𝑥 from improved subresultant algorithm
Output: 𝑟0 𝑥 , 𝑟1 𝑥 , ⋯ , 𝑟𝑙 𝑥
1. When𝑘 ∈ 𝐼𝑙, 𝑟′𝑘 𝑥 = 𝑟𝑙 𝑥 𝑥𝑛−𝑘
,𝑖 ← 𝑙 − 1
2.When 𝑘 ∈ 𝐼𝑖
2.1 Set Compute 𝜙 and 𝜓, such that 𝜓𝑙𝑐(𝑟𝑖) + 𝜙𝑙𝑐(𝑟𝑖+1) = gcd 𝑙𝑐 𝑟𝑖 , 𝑙𝑐 𝑟𝑖+1
2.2 Set 𝑟𝑖 𝑥 = 𝜓𝑟𝑖 𝑥 + 𝜙𝑟𝑖+1 𝑥 𝑥𝛿𝑖
2.3 If 𝑙𝑐 𝑟𝑛−𝑛𝑖
= 1, set 𝑟𝑗 𝑥 = 𝑟𝑛−𝑛𝑖
𝑥 𝑥𝑛−𝑛𝑖−𝑗
, 𝑗 = 1, ⋯ , 𝑛 − 𝑛𝑖, go to Step 3; otherwise 𝑟𝑘 𝑥 =
𝑟𝑛−𝑛𝑖
𝑥 𝑥𝑛−𝑛𝑖−𝑘
, 𝑖 ← 𝑙 − 1
2.4If 𝑖 > 0, go to Step 2, otherwise go to Step 3
3.Return𝑟0 𝑥 , 𝑟1 𝑥 , ⋯ , 𝑟𝑙 𝑥
We need to explain that Algorithm 2 is equivalent to the corresponding algorithm in [4] and we
just use polynomial representation notation to express the output instead of a matrix
representation notation in [4].
Then we will present some results of 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 and 𝑙𝑐(𝑟𝑖).
Lemma3. Let 𝑟1(𝑥), ⋯ , 𝑟𝑙(𝑥) be the polynomial remainder sequence obtained in improved
subresultant algorithm. Then 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑖−1 𝑥 for 0 ≤ 𝑖 ≤ 𝑙 − 1.
Proof. We prove this lemma by induction on 𝑖, 𝑖 = 0, … , 𝑙 − 1.

Vol 11, No 1/2/3/4, November 2022
12
Suppose that 𝑯 is the Hermite Normal Form over the ideal lattice ℒ generated by 𝑔(𝑥) ∈
ℤ[𝑥]/ 𝑓(𝑥) , and 𝑟𝑖(𝑥) belongs to ℒ. When 𝑖 = 0, because 𝑟0 𝑥 generates the ideal lattice ℒ ,
then all the vectors in ℒ can be divided exactly by 𝑐𝑜𝑛𝑡(𝑟0 𝑥 ).
Next we suppose that when 𝑖 ≤ 𝑘 − 1, 𝑐𝑜𝑛𝑡 𝑟𝑖−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 , then we need to show that
𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘+1 𝑥 .
Consider the 𝑘 + 1 -th equation in improved subresultant algorithm,
𝛼𝑘+1𝑟𝑘−1 𝑥 = 𝑞𝑘+1 𝑥 𝑟𝑘 𝑥 + 𝛽𝑘+1 𝑥 𝑟𝑘+1 𝑥 ,
then we know 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1+1
|𝛽𝑘+1𝑟𝑘+1 𝑥 . Because
𝑡𝑘+1 𝑥 = (𝛼𝑘+1𝑡𝑘−1 𝑥 − 𝑞𝑘+1 𝑥 𝑡𝑘 𝑥 ) 𝛽𝑘+1,
𝛽𝑘+1 must contain a factor as the content of 𝛼𝑘+1𝑡𝑘−1 𝑥 − 𝑞𝑘+1 𝑥 𝑡𝑘 𝑥 . Also 𝛼𝑘+1 =
𝑙𝑐(𝑟𝑘)𝛿𝑘−1+1
, (𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1
)|𝑞𝑘+1(𝑥) due to Proposition 1. Based on the
assumption(𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 , so 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1
|𝛽𝑘+1.
If (𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 ∤ 𝑐𝑜𝑛𝑡 𝑟𝑘+1 𝑥 , then there exists a prime 𝑎 such that 𝑎|𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 and
𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1
|𝛽𝑘+1. We give 2 cases as following:
1) 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 ∤ 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 , which means that 𝑎 ∤ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 and 𝑎 ∤
𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥
. According to Proposition 1, we know 𝑟𝑘+1 𝑥 = ∆𝑛𝑘−1,𝑗 𝑥𝑗
𝛽𝑘+1
𝑛𝑘−1
𝑗=0 , here
∆𝑛𝑘−1,𝑗 is the determinant of the (𝛿𝑘−1 + 2) × (𝛿𝑘−1 + 2) matrix obtained by deleting the
last 𝑛𝑘 columns except column 𝑛𝑘−1 + 1 − 𝑗 from 𝑀 , 𝑗 = 0, … , 𝑛𝑘 − 1 . Because 𝑎 ∤
𝑟𝑘−1 𝑥
, there exits a 𝑗 > 1 such that 𝑎|𝑀𝑗 (𝑥), which means 𝑎|
𝑙𝑐 𝑟𝑘
𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
. Thus we
obtan 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1
|𝛼𝑘+1 . According to equation 𝑡𝑘+1 𝑥 =
(𝛼𝑘+1𝑡𝑘−1 𝑥 − 𝑞𝑘+1 𝑥 𝑡𝑘 𝑥 ) 𝛽𝑘+1,we have that
𝛿𝑘−1
|𝑞𝑘+1(𝑥).
𝛿𝑘−1
|𝛽𝑘+1𝑟𝑘+1(𝑥), which means we have
get𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘+1 𝑥 .
2) 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 . Because 𝛼𝑘+1 = 𝑙𝑐 𝑟𝑘
𝛿𝑘−1+1
, then we have result that
𝑐𝑜𝑛𝑡 𝑟𝑘(𝑥) 𝛿𝑘−1+1
|𝛼𝑘+1 , thus 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥
𝛿𝑘−1
|𝛼𝑘+1 . As the same
step in case 1, we still get 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘+1 𝑥 .
So in conclusion we obtain 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘+1 𝑥 . The proof is completed.
•
Lemma 4. Let 𝑟1(𝑥), ⋯ , 𝑟𝑙(𝑥) be the polynomial remainder sequence obtained in improved
resultant algorithm and 𝑟1(𝑥), ⋯ , 𝑟𝑙(𝑥) be the output of Algorithm 2. If
gcd 𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1(𝑥) = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 for 𝑖 ≤ 𝑙 − 1 , then 𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 .
Moreover, 𝑙𝑐 𝑟𝑖 |𝑟𝑖(𝑥).

Vol 11, No 1/2/3/4, November 2022
13
Proof. We notice that from Algorithm 2, 𝑟𝑖 𝑥 = 𝜓𝑟𝑖 𝑥 + 𝜙𝑟𝑖+1 𝑥 𝑥𝛿𝑖 , where 𝜓 and 𝜙
satisfy 𝜓𝑙𝑐(𝑟𝑖) + 𝜙𝑙𝑐(𝑟𝑖+1) = gcd 𝑙𝑐 𝑟𝑖 , 𝑙𝑐 𝑟𝑖+1 = 𝑙𝑐(𝑟𝑖) . If we already have
gcd 𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1(𝑥) = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 and 𝑐𝑜𝑛𝑡 𝑟𝑖+1 𝑥 = 𝑙𝑐 𝑟𝑖+1 , then we have 𝑙𝑐 𝑟𝑖 =
𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 .
When 𝑖 = 𝑙, this is a trivial result because 𝑙𝑐 𝑟𝑙 = 𝑟𝑙 𝑥 = 𝑐𝑜𝑛𝑡(𝑟𝑙 𝑥 ). So we know 𝑙𝑐 𝑟𝑙−1 =
𝑐𝑜𝑛𝑡 𝑟𝑙−1 𝑥 , 𝑙𝑐 𝑟𝑙−2 = 𝑐𝑜𝑛𝑡 𝑟𝑙−2 𝑥 , …,, and so on. Thus, if gcd 𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1(𝑥) =
𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 for 𝑖 ≤ 𝑙 − 1, then 𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 .
For the second part, according to the assumption, gcd 𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1(𝑥) = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 ,
then 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑖+1 𝑥 . Also 𝑟𝑖 𝑥 = 𝜓𝑟𝑖 𝑥 + 𝜙𝑟𝑖+1 𝑥 𝑥𝛿𝑖 , so 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 |𝑟𝑖 𝑥 .
Due to 𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 , we know that 𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 |𝑟𝑖 𝑥 , for 0 ≤ 𝑖 ≤ 𝑙.
•
Lemma 5. Let 𝑟1(𝑥), ⋯ , 𝑟𝑙(𝑥) be the polynomial remainder sequence obtained in improved
resultant algorithm. Then gcd 𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1(𝑥) = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 .
Proof. We prove this lemma by induction on 𝑖, 𝑖 = 1, … , 𝑙 − 1.
First, suppose that 𝑯 is the Hermite Normal Form of the ideal lattice ℒ generated by 𝑔(𝑥) ∈
ℤ[𝑥]/ 𝑓(𝑥) , and 𝑟𝑖(𝑥) belongs to ℒ. Denote
𝑙𝑐 𝑟𝑖
𝑙𝑐 𝐻𝑛−𝑛𝑖
as 𝛾𝑖 and 𝑑𝑖 = 𝑛 − 𝑛𝑖, here 𝑯𝑖(𝑥) is the
corresponding polynomial of the 𝑖 -th row, then 𝑟𝑖 𝑥 = 𝛾𝑖𝑯𝑑𝑖
𝑥 + 𝐴𝑖,𝑗 (𝑥)
𝑙
𝑗=𝑖+1 𝑯𝑑𝑗
(𝑥) ,
where deg⁡
(𝐴𝑖,𝑗 ) < 𝑛𝑖 − 𝑛𝑗 .
From Lemma 4, 𝑙𝑐 𝑯𝑑𝑗
|𝑯𝑑𝑗
(𝑥), for 𝑖 ≤ 𝑗 ≤ 𝑙 . So 𝑙𝑐 𝑯𝑑𝑖
|𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 . Because 𝑟𝑖 𝑥 =
𝑡𝑖(𝑥)𝑔(𝑥)𝑚𝑜𝑑𝑓(𝑥) belongs to ℒ and 𝑡𝑖(𝑥) is primitive, then gcd⁡
(𝛾𝑖, 𝐴𝑖,𝑖+1 𝑥 , … , 𝐴𝑖,𝑙 𝑥 ) = 1,
thus there exists some 𝑖 < 𝑘 ≤ 𝑙 , 𝑐𝑜𝑛𝑡
𝑟𝑖 𝑥
𝑙𝑐 𝑯𝑑𝑖
= gcd⁡
(𝛾𝑖,
𝑙𝑐 𝑯𝑑𝑘
𝑙𝑐 𝑯𝑑𝑖
) , which means that
𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 = gcd⁡
(𝑙𝑐 𝑟𝑖 , 𝑙𝑐(𝑯𝑑𝑘
)) . So every content of 𝑟𝑖 𝑥 must be a factor of 𝑯𝑛 .
Specially, we have 𝑐𝑜𝑛𝑡 𝑟𝑙−1 𝑥 = 𝑙𝑐(𝑟𝑙−1), which shows that the result holds for 𝑖 = 𝑙 − 1.
Now assume that for 𝑖 ≥ 𝑘 , we have gcd⁡
(𝑙𝑐 𝑟𝑖 , 𝑐𝑜𝑛𝑡 𝑟𝑖+1 𝑥 = 𝑐𝑜𝑛𝑡(𝑟𝑖 𝑥 ). Then from
Lemma 3, we have 𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 .
Next we consider 𝑘 − 1, from the Algorithm 2, gcd⁡
(𝑙𝑐 𝑟𝑘−1 , 𝑙𝑐 𝑟𝑘 ) = 𝑙𝑐(𝑟𝑘−1). Then because
𝑙𝑐 𝑟𝑖 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 for 𝑖 ≥ 𝑘, gcd⁡
(𝑙𝑐 𝑟𝑘−1 , 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 ) = 𝑙𝑐(𝑟𝑘−1). So we need to show
𝑙𝑐 𝑟𝑘−1 = 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 .
First, 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑙𝑐 𝑟𝑘−1 and according to the Lemma 3 we have divisibility relation,
𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘(𝑥) , so 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |gcd⁡
(𝑙𝑐 𝑟𝑘−1 , 𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 ) = 𝑙𝑐(𝑟𝑘−1) . We
suppose 𝑙𝑐 𝑟𝑘−1 = 𝑎 ∙ 𝑐𝑜𝑛𝑡(𝑟𝑘−1 𝑥 ) for a prime 𝑎 . According to Lemma 4, 𝑙𝑐 𝑟𝑖 =
𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 = 𝑐𝑜𝑛𝑡 𝑟𝑖 𝑥 for 𝑖 ≥ 𝑘, so we have 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘−1(𝑥) . Also the step
diminishes the leading coefficient and 𝑙𝑐 𝑟𝑘−1 |𝑙𝑐 𝑟𝑘−1 , then 𝑐𝑜𝑛𝑡 𝑟𝑘−1(𝑥) ≤ 𝑐𝑜𝑛𝑡(𝑟𝑘−1 𝑥 ).
So 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 = 𝑐𝑜𝑛𝑡 𝑟𝑘−1(𝑥) .

Vol 11, No 1/2/3/4, November 2022
14
Consider the 𝑘-th equation in improved sub resultant algorithm,
𝛼𝑘𝑟𝑘−2 𝑥 = 𝑞𝑘 𝑥 𝑟𝑘−1 𝑥 + 𝛽𝑘𝑟𝑘 𝑥
here 𝛼𝑘 = 𝑙𝑐 𝑟𝑘−1
𝛿𝑘−2+1
.
Because 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑙𝑐(𝑟𝑘−1) and 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑐𝑜𝑛𝑡 𝑟𝑘 𝑥 , we know that 𝑎 ∙
𝛿𝑘−2+1
|𝛼𝑘 and 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥 |𝑟𝑘 𝑥 , which means, if we divide the
equation above by 𝜇 = 𝑎 ∙ 𝑐𝑜𝑛𝑡 𝑟𝑘−1 𝑥
𝛿𝑘−2+1
𝑐𝑜𝑛𝑡(𝑟𝑘−2 𝑥 ), then
𝛼𝑘𝑟𝑘−2 𝑥
𝜇
and
𝛽𝑘 𝑟𝑘 𝑥
𝜇
both
belong to ℤ[𝑥], while
𝑞𝑘 (𝑥)𝑟𝑘−1 𝑥
𝜇
doesn't. This is a contradiction. So the proof is completed.
•
Using the results above, we realize that 𝑙𝑐 𝑡𝑖 is related to 𝜏𝑖 which is unknown yet. We tried
some equations and found the following equation,
gcd 𝑙𝑐 𝑡𝑖 , 𝑙𝑐 𝑟𝑖−1 = 𝑔𝑐𝑑
, 𝑙𝑐 𝑟𝑖−1
for 𝑖 = 0,1,2, ⋯ , 𝑙 .Also in our experiments, the equation above holds with extremely high
probability.
4. A PROBABILISTIC ALGORITHM FOR COMPUTATION OF POLYNOMIAL
GREATEST COMMON
In this section, we give a probabilistic subresultant algorithm by applying the results in the last
section. We need to emphasis that the algorithm is not deterministic yet. The detail of the
algorithm is presented as following.
Algorithm 3 Probabilistic Subresultant Algorithm
Input: two polynomials 𝑓(𝑥), 𝑔(𝑥) ∈ ℤ[𝑥] with degree 𝑛 and 𝑚 respectively and 𝑓(𝑥) is monic and
irreducible
Output: Probabilistic subresultant PRS, 𝑟0 𝑥 , 𝑟1 𝑥 , ⋯
1.[Initialize] 𝑙 ← 𝑕 ← 1, 𝑢1(𝑥) ← 𝑓(𝑥),𝑢2(𝑥) ← 𝑔(𝑥),𝑖 ← 1
2.Compute 𝑙𝑐(𝑢2)𝛿+1
𝑢1(𝑥) − 𝑞(𝑥)𝑢2(𝑥) = 𝑟(𝑥), here 𝑑𝑒𝑔(𝑟) < 𝑑𝑒𝑔(𝑢2), 𝛿 = 𝑑𝑒𝑔(𝑢1) − 𝑑𝑒𝑔(𝑢2)
3.𝑢(𝑥) ← 𝑢1(𝑥), 𝑢1(𝑥) ← 𝑢2(𝑥), 𝑢2(𝑥) ← 𝑟(𝑥)
4.When 𝑑𝑒𝑔(𝑢2) ≠ 0,
4.1 𝑙 ← 𝑙𝑐(𝑢2), 𝑕 ← 𝑙𝛿
𝑕1−𝛿
4.2 𝜏 ← gcd(𝑕, 𝑐𝑜𝑛𝑡(𝑢2(𝑥))), 𝜏′ ← gcd(𝑙𝑐(𝑢)/𝑐𝑜𝑛𝑡(𝑢(𝑥)), 𝑐𝑜𝑛𝑡(𝑢(𝑥)))
4.3 𝜏 ←/𝜏′, 𝑟𝑖(𝑥) ← 𝑟(𝑥)/(𝑙𝑕𝛿
𝜏)
4.4 𝛿 = 𝑑𝑒𝑔(𝑢1) − 𝑑𝑒𝑔(𝑢2)
4.5 Compute 𝑙𝑐(𝑢2)𝛿+1
𝑢1(𝑥) − 𝑞(𝑥)𝑢2(𝑥) = 𝑟(𝑥), 𝑑𝑒𝑔(𝑟) < 𝑑𝑒𝑔(𝑢2), 𝛿 = 𝑑𝑒𝑔(𝑢1) − 𝑑𝑒𝑔(𝑢2)
4.6 𝑢(𝑥) ← 𝑢1(𝑥), 𝑢1(𝑥) ← 𝑢2(𝑥), 𝑢2(𝑥) ← 𝑟(𝑥)/(𝑙𝑕𝛿
)
4.7 𝑖 ← 𝑖 + 1
5.[Return] 𝑟0 𝑥 , 𝑟1 𝑥 , ⋯
For the often-used polynomials in ideal lattice-based cryptography 𝑥𝑛
+ 1 and 𝑥𝑛
− 𝑥 − 1, here
𝑛 is a power of 2, we give the experiment results.

Vol 11, No 1/2/3/4, November 2022
15
Our experiments were performed on a PC (Intel(R) Core(TM) i7, 3.4GHz, 2G RAM) using C
language without any optimization. For each polynomial, we sample 10000 examples randomly
with coefficients in the range [-20, 20] with degree 𝑛 32, 64 and 128. The whole correctness is
presented below.
Polynomial 𝑥𝑛
+ 1 𝑥𝑛
− 𝑥 − 1
Correctness 97.88% 99.73%
In addition, to measure the efficiency of our algorithm, we made experiment between the
previous best extended Euclidean algorithm in [7] with our algorithm. In the same platform, we
use 𝑓 𝑥 = 𝑥𝑛
− 𝑥 − 1 and the average time to output the desired remainders of each algorithm
is presented below. The time is From the table, the time of our algorithm is about 1/3 of the time
in [7].
degree
Algorithm
32 64 128
Our Algorithm 0.006s 0.07s 0.59s
Algorithm in [7] 0.016s 0.23s 1.814s
5. CONCLUSIONS
In this paper, we give some results about the contents and small factors of remainders during the
extended Euclidean algorithm of polynomials. By applying these results, we proposed a
probabilistic subresultant algorithm which can output correct remainders with overwhelming
probability.
Due to the case of failure, our further research will focus on the exact expression of each 𝜏𝑖 and
relation between 𝑙𝑐 𝑡𝑖 and cont(𝑟
𝑗 (𝑥)) to obtain a determinisitic improved subresultant
algorithm, which will improve the efficiency of computing the public key in lattice-base
cryptography.
REFERENCES
[1] G.E. Collins. Subresultants and reduced polynomial remainder sequences. J. ACM, 14(1): 128--142
(1967)
[2] D.E. Knuth. The art of computer programming. Seminumerical Algorithms. vol. 2, 3rd Edition, 1998
(1st Edition, 1969)
[3] W.S. Brown. The subresultant PRS algorithm. ACM Trans. Math. Software, 4(3):237--249 (1978)
[4] Y. Zhang , R.Z. Liu, D.D. Lin. Fast Triangularization of Ideal Latttice Basis. Journal of Electronics
and Information Technology, 42(1): 98-104 (2020)
[5] Y. Zhang , R.Z. Liu, D.D. Lin. Improved Key Generation Algorithm for Gentry's Fully
Homomorphic Encryption Scheme. ICISC: 97-111 (2018)
[6] J Gathen, T Lücking. Subresultants Revisited. Latin American Symposium on Theoretical
Informatics, LNCS 1776: 318–342
[7] VON ZUR GATHEN J and GARHARD J. Modern Computer Algebra. 3rd ed. Cambridge:
Cambridge University Press, 2013: 313–332.

A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR WITH SMALLER FACTORS

More Related Content

Similar to A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR WITH SMALLER FACTORS (20)

Recently uploaded (20)

A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR WITH SMALLER FACTORS