Divide-and-conquer recurrences and the Akra-Bazzi theorem #
A divide-and-conquer recurrence is a function T : ℕ → ℝ
that satisfies a recurrence relation of
the form T(n) = ∑_{i=0}^{k-1} a_i T(r_i(n)) + g(n)
for large enough n
, where r_i(n)
is some
function where ‖r_i(n) - b_i n‖ ∈ o(n / (log n)^2)
for every i
, the a_i
's are some positive
coefficients, and the b_i
's are reals ∈ (0,1)
. (Note that this can be improved to
O(n / (log n)^(1+ε))
, this is left as future work.) These recurrences arise mainly in the
analysis of divide-and-conquer algorithms such as mergesort or Strassen's algorithm for matrix
multiplication. This class of algorithms works by dividing an instance of the problem of size n
,
into k
smaller instances, where the i
'th instance is of size roughly b_i n
, and calling itself
recursively on those smaller instances. T(n)
then represents the running time of the algorithm,
and g(n)
represents the running time required to actually divide up the instance and process the
answers that come out of the recursive calls. Since virtually all such algorithms produce instances
that are only approximately of size b_i n
(they have to round up or down at the very least), we
allow the instance sizes to be given by some function r_i(n)
that approximates b_i n
.
The Akra-Bazzi theorem gives the asymptotic order of such a recurrence: it states that
T(n) ∈ Θ(n^p (1 + ∑_{u=0}^{n-1} g(n) / u^{p+1}))
,
where p
is the unique real number such that ∑ a_i b_i^p = 1
.
Main definitions and results #
AkraBazziRecurrence T g a b r
: the predicate stating thatT : ℕ → ℝ
satisfies an Akra-Bazzi recurrence with parametersg
,a
,b
andr
as above.GrowsPolynomially
: The growth condition thatg
must satisfy for the theorem to apply. It roughly states thatc₁ g(n) ≤ g(u) ≤ c₂ g(n)
, for u between b*n and n for any constantb ∈ (0,1)
.sumTransform
: The transformation which turns a functiong
inton^p * ∑ u in Finset.Ico n₀ n, g u / u^(p+1)
.asympBound
: The asymptotic bound satisfied by an Akra-Bazzi recurrence, namelyn^p (1 + ∑ g(u) / u^(p+1))
isTheta_asympBound
: The main result stating thatT(n) ∈ Θ(n^p (1 + ∑_{u=0}^{n-1} g(n) / u^{p+1}))
Implementation #
Note that the original version of the theorem has an integral rather than a sum in the above
expression, and first considers the T : ℝ → ℝ
case before moving on to ℕ → ℝ
. We prove the
above version with a sum, as it is simpler and more relevant for algorithms.
TODO #
- Specialize this theorem to the very common case where the recurrence is of the form
T(n) = ℓT(r_i(n)) + g(n)
whereg(n) ∈ Θ(n^t)
for somet
. (This is often called the "master theorem" in the literature.) - Add the original version of the theorem with an integral instead of a sum.
References #
- Mohamad Akra and Louay Bazzi, On the solution of linear recurrence equations
- Tom Leighton, Notes on better master theorems for divide-and-conquer recurrences
- Manuel Eberl, Asymptotic reasoning in a proof assistant
Definition of Akra-Bazzi recurrences #
This section defines the predicate AkraBazziRecurrence T g a b r
which states that T
satisfies the recurrence
T(n) = ∑_{i=0}^{k-1} a_i T(r_i(n)) + g(n)
with appropriate conditions on the various parameters.
An Akra-Bazzi recurrence is a function that satisfies the recurrence
T n = (∑ i, a i * T (r i n)) + g n
.
- n₀ : ℕ
Point below which the recurrence is in the base case
- n₀_gt_zero : 0 < self.n₀
n₀
is always> 0
- a_pos : ∀ (i : α), 0 < a i
The
a
's are nonzero - b_pos : ∀ (i : α), 0 < b i
The
b
's are nonzero - b_lt_one : ∀ (i : α), b i < 1
The b's are less than 1
- g_nonneg : ∀ x ≥ 0, 0 ≤ g x
g
is nonnegative - g_grows_poly : AkraBazziRecurrence.GrowsPolynomially g
g
grows polynomially - h_rec : ∀ (n : ℕ), self.n₀ ≤ n → T n = (Finset.sum Finset.univ fun (i : α) => a i * T (r i n)) + g ↑n
The actual recurrence
- T_gt_zero' : ∀ n < self.n₀, 0 < T n
Base case:
T(n) > 0
whenevern < n₀
The
r
's always reducen
- dist_r_b : ∀ (i : α), (fun (n : ℕ) => ↑(r i n) - b i * ↑n) =o[Filter.atTop] fun (n : ℕ) => ↑n / Real.log ↑n ^ 2
The
r
's approximate theb
's
Instances For
Smallest b i
Equations
- AkraBazziRecurrence.min_bi b = Classical.choose (_ : ∃ (x₀ : α), ∀ (x : α), b x₀ ≤ b x)
Instances For
Largest b i
Equations
- AkraBazziRecurrence.max_bi b = Classical.choose (_ : ∃ (x₀ : α), ∀ (x : α), b x ≤ b x₀)
Instances For
Smoothing function #
We define ε
as the "smoothing function" fun n => 1 / log n
, which will be used in the form of a
factor of 1 ± ε n
needed to make the induction step go through.
This is its own definition to make it easier to switch to a different smoothing function.
For example, choosing 1 / log n ^ δ
for a suitable choice of δ
leads to a slightly tighter
theorem at the price of a more complicated proof.
This part of the file then proves several properties of this function that will be needed later in the proof.
The "smoothing function" is defined as 1 / log n
. This is defined as an ℝ → ℝ
function
as opposed to ℕ → ℝ
since this is more convenient for the proof, where we need to e.g. take
derivatives.
Equations
Instances For
Akra-Bazzi exponent p
#
Every Akra-Bazzi recurrence has an associated exponent, denoted by p : ℝ
, such that
∑ a_i b_i^p = 1
. This section shows the existence and uniqueness of this exponent p
for any
R : AkraBazziRecurrence
, and defines R.asympBound
to be the asymptotic bound satisfied by R
,
namely n^p (1 + ∑_{u < n} g(u) / u^(p+1))
.
The function x ↦ ∑ a_i b_i^x is injective. This implies the uniqueness of p
.
The exponent p
associated with a particular Akra-Bazzi recurrence.
Equations
Instances For
The sum transform #
This section defines the "sum transform" of a function g
as
∑ u in Finset.Ico n₀ n, g u / u^(p+1)
,
and uses it to define asympBound
as the bound satisfied by an Akra-Bazzi recurrence.
Several properties of the sum transform are then proven.
The transformation which turns a function g
into
n^p * ∑ u in Finset.Ico n₀ n, g u / u^(p+1)
.
Equations
- AkraBazziRecurrence.sumTransform p g n₀ n = ↑n ^ p * Finset.sum (Finset.Ico n₀ n) fun (u : ℕ) => g ↑u / ↑u ^ (p + 1)
Instances For
The asymptotic bound satisfied by an Akra-Bazzi recurrence, namely
n^p (1 + ∑_{u < n} g(u) / u^(p+1))
.
Equations
- AkraBazziRecurrence.asympBound g a b n = ↑n ^ AkraBazziRecurrence.p a b + AkraBazziRecurrence.sumTransform (AkraBazziRecurrence.p a b) g 0 n
Instances For
Technical lemmas #
The next several lemmas are technical lemmas leading up to rpow_p_mul_one_sub_smoothingFn_le
and
rpow_p_mul_one_add_smoothingFn_ge
, which are key steps in the main proof.
Main proof #
This final section proves the Akra-Bazzi theorem.
The main proof of the upper bound part of the Akra-Bazzi theorem. The factor
1 - ε n
does not change the asymptotic order, but is needed for the induction step to go
through.
The main proof of the lower bound part of the Akra-Bazzi theorem. The factor
1 + ε n
does not change the asymptotic order, but is needed for the induction step to go
through.
The Akra-Bazzi theorem: T ∈ O(n^p (1 + ∑_u^n g(u) / u^{p+1}))
The Akra-Bazzi theorem: T ∈ Ω(n^p (1 + ∑_u^n g(u) / u^{p+1}))
The Akra-Bazzi theorem: T ∈ Θ(n^p (1 + ∑_u^n g(u) / u^{p+1}))