{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bgoujaud/PEPit/blob/master/ressources/demo/PEPit_demo.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# PEPit : numerical examples of worst-case analyses" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook provides:\n", "- a simple example illustrating how to obtain a worst-case guarantee for **gradient descent** using the PEPit package,\n", "- worst-case analyses for 4 more advanced examples, for which we compare analytical and numerical worst-case guarantees, respectively obtained in the literature and by PEPit. \n", "\n", "In short, PEPit is a package enabling **computer-assisted worst-case analyses** of first-order optimization methods. The key underlying idea is to cast the problem of performing a worst-case analysis, often referred to as a performance estimation problem (PEP), as a semidefinite program (SDP) which can be solved numerically. For doing that, the package users are only required to write first-order methods nearly as they would have implemented them. The package then takes care of the SDP modelling parts, and the worst-case analysis is performed numerically via a standard solver.\n", "\n", "\n", "PEPit can be installed following these [instructions](https://pypi.org/project/PEPit/), and a quickstart is available in its [documentation](https://pepit.readthedocs.io/en/latest/). More information can be found in the [PEPit reference paper](https://arxiv.org/pdf/2201.04040.pdf). \n", "\n", "We refer to [this blog post](https://francisbach.com/computer-aided-analyses/) and the references therein for a more mathematical introduction to performance estimation problems." ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "\n", "## Table of Contents\n", "- [1 Installing PEPit](#1-Installing-PEPit)\n", "- [2 How to obtain a worst-case guarantee for GD using PEPit](#2-How-to-obtain-a-worst-case-guarantee-for-GD-using-PEPit)\n", " - [2.1 Imports](#2.1-Imports)\n", " - [2.2 Initialization of PEPit](#2.2-Initialization-of-PEPit)\n", " - [2.3 Choose parameters values](#2.3-Choose-parameters-values)\n", " - [2.4 Specifying the problem class](#2.4-Specifying-the-problem-class)\n", " - [2.5 Algorithm initialization](#2.5-Algorithm-initialization)\n", " - [2.6 Algorithm implementation](#2.6-Algorithm-implementation)\n", " - [2.7 Setting up a performance measure](#2.7-Setting-up-a-performance-measure)\n", " - [2.8 Solving the PEP](#2.8-Solving-the-PEP)\n", " - [2.9 Analytical upper bound comparison](#2.9-Comparing-to-the-an-(established)-analytical-upper-bound-(worst-case-guarantee)-provided-by-the-literature)\n", " - [2.10 Conclusion](#2.10-Conclusion)\n", "- [3 Comparison of analytical and numerical worst-case guarantees](#3-Comparison-of-analytical-(obtained-in-the-literature)-and-numerical-(obtained-by-PEPit)-worst-case-guarantees-on-4-more-advanced-methods)\n", " - [3.1 Example 1: GD with fixed step size](#3.1-Example-1-:-Gradient-descent-with-fixed-step-size-in-contraction)\n", " - [3.1.1 Worst-case guarantees as functions of the iteration count](#3.1.1-Worst-case-guarantees-as-functions-of-the-iteration-count)\n", " - [3.1.2 Worst-case guarantees as functions of the step size](#3.1.2-Worst-case-guarantees-as-functions-of-the-step-size)\n", " - [3.2 Example 2: Accelerated gradient method](#3.2-Example-2-:-An-accelerated-gradient-method-for-strongly-convex-objectives)\n", " - [3.3 Example 3: Accelerated Douglas-Rachford splitting](#3.3-Example-3-:-An-accelerated-Douglas-Rachford-splitting)\n", " - [3.4 Example 4: point-SAGA](#3.4-Example-4-:-point-SAGA)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1 Installing PEPit" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# If PEPit is not installed yet, you can run this cell.\n", "!pip3 install pepit;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
↩ Back to TOC
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2 How to obtain a worst-case guarantee for GD using PEPit" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section, we provide a step-by-step tutorial on how to use PEPit to compute a worst-case guarantee for gradient descent.\n", "\n", "That is, we consider the minimization problem\n", "\n", "$$\\min_{x} f(x),$$\n", "\n", "where $f$ os an $L$-smooth convex function which is minimized at $x_*$. We denote by $x_n$ the output of gradient descent with step-size $\\gamma$ after $n\\in\\mathbb{N}$ iterations, starting from $x_0$. We perform a worst-case analysis in the following sense: we compute the smallest possible value of $\\tau(L,\\gamma,n)$ such that\n", "\n", "$$ f(x_{n})-f(x_*) \\leqslant\\ \\tau(L,\\gamma,n)\\ \\ \\|x_0-x_*\\|^2_2$$\n", "\n", "is valid for all $L$-smooth convex function $f$ and initial iterate $x_0$. Computing the smallest possible such $\\tau(L,\\gamma,n)$ is equivalent to computing the worst-case value of $f(x_{n})-f(x_*)$ under the constraint $\\| x_{0}-x_*\\|_2^2\\leqslant 1$.\n", "\n", "PEPit formulates the problem of computing $\\tau(L,\\gamma,n)$ as an SDP which can be solved numerically. In what follows, we describe the few lines that a user has to provide for generating the numerical worst-case analysis of GD using PEPit.\n", "\n", "In general, performing a worst-case analysis of a first-order method usually relies on **four main ingredients**: \n", "- a class of problems (containing the assumptions on the function to be minimized),\n", "- a first-order method (to be analyzed), \n", "- a performance measure (measuring the quality of the output of the algorithm under consideration),\n", "- an initial condition (measuring the quality of the initial iterate). \n", "\n", "PEPit requires the user to specify a choice for each of those four elements." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.1 Imports\n", "Before anything else, we have to import the PEP and the classes of functions of interest." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "from PEPit import PEP\n", "from PEPit.functions import SmoothStronglyConvexFunction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.2 Initialization of PEPit\n", "Then, we initialize a PEP object. This object allows manipulating the forthcoming ingredients of the PEP, such as functions and iterates." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Instantiate PEP\n", "problem = PEP()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.3 Choose parameters values\n", "\n", "For the sake of the example, we pick some simple values for the problem class and algorithmic parameters, for which we perform the worst-case analysis below. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Specify values for the smoothness parameter L, number of iterations n, and the step size gamma.\n", "L = 3\n", "n = 4\n", "gamma = 1 / L" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### 2.4 Specifying the problem class\n", "Next, we specify our assumptions on the **class of functions** containing the function to be optimized, and instantiate a corresponding object. Here, we consider an $L$-smooth convex function. \n", "\n", "
*Remark*: PEPit can handle various function classes [(documentation)](https://pepit.readthedocs.io/en/latest/api/functions_and_operators.html).\n", "\n", "Note: we use the ```SmoothStronglyConvexFunction``` class with $\\mu=0$ here. Alternatively, one could use ```SmoothConvexFunction```." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Declare an L-smooth mu-strongly convex function named \"func\"\n", "func = problem.declare_function(\n", " SmoothStronglyConvexFunction,\n", " mu=0, # Strong convexity param.\n", " L=L) # Smoothness param." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also define $x_*$ as an optimal point and $f_*$ as the optimal value." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Start by defining an optimal point xs = x_* and corresponding function value fs = f_*\n", "xs = func.stationary_point()\n", "fs = func(xs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.5 Algorithm initialization\n", "\n", "Third, we can instantiate the starting points for the two gradient methods that we will run, and specify an **initial condition** on those points. \n", "To this end, a starting point $x_0$ is introduced, and a bound on the initial distance between those points is specified as $\\|x_0-x_*\\|^2\\leqslant 1$.\n", "\n", "
*Remark*: PEPit can handle various initial constraints, for example any linear combination of $f(x_0)-f_*$, $\\|x_0-x_*\\|^2$ or $\\|\\nabla f(x_0)\\|^2$.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Then define the starting point x0 of the algorithm\n", "x0 = problem.set_initial_point()\n", "\n", "# Set the initial constraint that is the distance between x0 and x^*\n", "problem.set_initial_condition((x0 - xs) ** 2 <= 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.6 Algorithm implementation\n", "\n", "In this fourth step, we specify the **algorithm** in a natural format. In this example, we simply use the iterates (which are PEPit objects) as if we had to implement gradient descent in practice using a simple loop. Gradient descent with fixed step size $\\gamma$ may be described as follows, for $t \\in \\{0,1, \\ldots, n-1\\}$\n", "\\begin{equation}\n", "x_{t+1} = x_t - \\gamma \\nabla f(x_t).\n", "\\end{equation}\n", "\n", "
*Remark*: PEPit can handle most first order algorithms that rely on some simple [oracles.](https://pepit.readthedocs.io/en/latest/api/steps.html)
More than 50 examples (for deterministic, stochastic, and/or composite optimization) are provided in the [documentation](https://pepit.readthedocs.io/en/latest/examples.html)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Run n steps of gradient descent with step-size gamma\n", "x = x0\n", "for _ in range(n):\n", " x = x - gamma * func.gradient(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.7 Setting up a performance measure\n", "\n", "It is crucial for the worst-case analysis to specify the **performance metric** for which we wish to compute a worst-case performance. In this example, we wish to compute the worst-case value of $f(x_n)-f_*$, which we specify as follows. \n", "\n", "
*Remark*: PEPit can handle various performance metrics, for example any linear combination of $f(x_n)-f_*$, $\\|x_n-x_*\\|^2$ or $\\|\\nabla f(x_n)\\|^2$.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the performance metric to the function values accuracy\n", "problem.set_performance_metric(func(x) - fs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.8 Solving the PEP\n", "\n", "The last natural stage in the process is to solve the corresponding PEP. This is done via the following line, which will ask PEPit to perform the modelling steps and to call an appropriate SDP solver (which should be installed beforehand) to perform the worst-case analysis.\n", "\n", "The solver is [CVXPY](https://www.cvxpy.org/) (the default solver being [SCS](https://web.stanford.edu/~boyd/papers/scs.html) which might not be the most appropriate solvers for solving SDPs; we advise installing [MOSEK](https://www.mosek.com/) if possible, for improved numerical performances)." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(PEPit) Setting up the problem: size of the Gram matrix: 7x7\n", "(PEPit) Setting up the problem: performance measure is the minimum of 1 element(s)\n", "(PEPit) Setting up the problem: Adding initial conditions and general constraints ...\n", "(PEPit) Setting up the problem: initial conditions and general constraints (1 constraint(s) added)\n", "(PEPit) Setting up the problem: interpolation conditions for 1 function(s)\n", "\t\t\tFunction 1 : Adding 30 scalar constraint(s) ...\n", "\t\t\tFunction 1 : 30 scalar constraint(s) added\n", "(PEPit) Setting up the problem: additional constraints for 0 function(s)\n", "(PEPit) Compiling SDP\n", "(PEPit) Calling SDP solver\n", "(PEPit) Solver status: optimal (wrapper:cvxpy, solver: SCS); optimal value: 0.16666634632022087\n", "\u001b[96m(PEPit) Postprocessing: solver's output is not entirely feasible (smallest eigenvalue of the Gram matrix is: -2.6e-07 < 0).\n", " Small deviation from 0 may simply be due to numerical error. Big ones should be deeply investigated.\n", " In any case, from now the provided values of parameters are based on the projection of the Gram matrix onto the cone of symmetric semi-definite matrix.\u001b[0m\n", "(PEPit) Primal feasibility check:\n", "\t\tThe solver found a Gram matrix that is positive semi-definite up to an error of 2.6031562588658637e-07\n", "\t\tAll the primal scalar constraints are verified up to an error of 6.94474186248295e-07\n", "(PEPit) Dual feasibility check:\n", "\t\tThe solver found a residual matrix that is positive semi-definite\n", "\t\tAll the dual scalar values associated with inequality constraints are nonnegative\n", "(PEPit) The worst-case guarantee proof is perfectly reconstituted up to an error of 3.3690239638641244e-06\n", "(PEPit) Final upper bound (dual): 0.16666732807401075 and lower bound (primal example): 0.16666634632022087 \n", "(PEPit) Duality gap: absolute: 9.817537898748618e-07 and relative: 5.8905340613190735e-06\n" ] } ], "source": [ "# Solve the PEP with verbose = 1\n", "verbose = 1\n", "pepit_tau = problem.solve(verbose=verbose)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.9 Comparing to the an (established) analytical upper bound (worst-case guarantee) provided by the literature\n", "\n", "We can now compare to a **analytical value**, provided in [1, Theorem 1]. When $\\gamma \\leqslant \\frac{1}{L}$, the analytical value for the exact worst case guarantee can be found is:\n", "$$f(x_n)-f_* \\leqslant \\frac{L||x_0-x_\\star||^2}{4nL\\gamma+2}$$\n", "which is achieved on some Huber loss functions.\n", "\n", "\n", "[1] Y. Drori, M. Teboulle (2014). [Performance of first-order methods for smooth convex minimization: a novel approach](https://arxiv.org/pdf/1206.3209.pdf). Mathematical Programming 145(1–2), 451–482.\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "*** Worst-case performance of gradient descent with fixed step-sizes ***\n", "\tPEPit guarantee:\t f(x_n)-f_* <= 0.166667 ||x_0 - x_*||^2\n", "\tTheoretical guarantee:\t f(x_n)-f_* <= 0.166667 ||x_0 - x_*||^2\n" ] } ], "source": [ "# Compute theoretical guarantee (for comparison)\n", "theoretical_tau = L / (4 * n * L * gamma + 2)\n", "\n", "# Print conclusion if required\n", "if verbose != -1:\n", " print('*** Worst-case performance of gradient descent with fixed step-sizes ***')\n", " print('\\tPEPit guarantee:\\t f(x_n)-f_* <= {:.6} ||x_0 - x_*||^2'.format(pepit_tau))\n", " print('\\tTheoretical guarantee:\\t f(x_n)-f_* <= {:.6} ||x_0 - x_*||^2'.format(theoretical_tau))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.10 Conclusion\n", "For the parameters specified above, the worst-case guarantee obtained numerically by PEPit matches the analytical value of the best known (tight) worst-case guarantee.\n", "\n", "The following code is provided as one of the 50+ examples in the package, see [`GradientMethod`](https://pepit.readthedocs.io/en/latest/examples/a.html#gradient-descent). \n", "\n", "It is possible to directly re-run all the cells above using the ````wc_gradient_descent```` function." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(PEPit) Setting up the problem: size of the Gram matrix: 7x7\n", "(PEPit) Setting up the problem: performance measure is the minimum of 1 element(s)\n", "(PEPit) Setting up the problem: Adding initial conditions and general constraints ...\n", "(PEPit) Setting up the problem: initial conditions and general constraints (1 constraint(s) added)\n", "(PEPit) Setting up the problem: interpolation conditions for 1 function(s)\n", "\t\t\tFunction 1 : Adding 30 scalar constraint(s) ...\n", "\t\t\tFunction 1 : 30 scalar constraint(s) added\n", "(PEPit) Setting up the problem: additional constraints for 0 function(s)\n", "(PEPit) Compiling SDP\n", "(PEPit) Calling SDP solver\n", "(PEPit) Solver status: optimal (wrapper:cvxpy, solver: SCS); optimal value: 0.16666634632022087\n", "\u001b[96m(PEPit) Postprocessing: solver's output is not entirely feasible (smallest eigenvalue of the Gram matrix is: -2.6e-07 < 0).\n", " Small deviation from 0 may simply be due to numerical error. Big ones should be deeply investigated.\n", " In any case, from now the provided values of parameters are based on the projection of the Gram matrix onto the cone of symmetric semi-definite matrix.\u001b[0m\n", "(PEPit) Primal feasibility check:\n", "\t\tThe solver found a Gram matrix that is positive semi-definite up to an error of 2.6031562588658637e-07\n", "\t\tAll the primal scalar constraints are verified up to an error of 6.94474186248295e-07\n", "(PEPit) Dual feasibility check:\n", "\t\tThe solver found a residual matrix that is positive semi-definite\n", "\t\tAll the dual scalar values associated with inequality constraints are nonnegative\n", "(PEPit) The worst-case guarantee proof is perfectly reconstituted up to an error of 3.3690239638641244e-06\n", "(PEPit) Final upper bound (dual): 0.16666732807401075 and lower bound (primal example): 0.16666634632022087 \n", "(PEPit) Duality gap: absolute: 9.817537898748618e-07 and relative: 5.8905340613190735e-06\n", "*** Example file: worst-case performance of gradient descent with fixed step-sizes ***\n", "\tPEPit guarantee:\t f(x_n)-f_* <= 0.166667 ||x_0 - x_*||^2\n", "\tTheoretical guarantee:\t f(x_n)-f_* <= 0.166667 ||x_0 - x_*||^2\n" ] } ], "source": [ "from PEPit.examples.unconstrained_convex_minimization import wc_gradient_descent\n", "pepit_tau, theoretical_tau = wc_gradient_descent(L=L, gamma=1 / L, n=4, verbose=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
↩ Back to TOC
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3 Comparison of analytical (obtained in the literature) and numerical (obtained by PEPit) worst-case guarantees on 4 more advanced methods\n", "\n", "In this section, we compare the worst-case guarantees obtained with PEPit and to reference worst-case guarantees for four algorithms:\n", "- **gradient descent** with fixed step size (in terms of contraction rate),\n", "- an **accelerated gradient** method (see for example [this ressource](https://arxiv.org/pdf/2101.09545.pdf), Algorithm 16) for strongly convex objectives,\n", "- an **accelerated Douglas-Rachford** splitting (see [this ressource](https://arxiv.org/pdf/1407.6723.pdf), Section IV),\n", "- **point-SAGA** (see [this ressource](https://proceedings.neurips.cc/paper/2016/file/4f6ffe13a5d75b2d6a3923922b3922e5-Paper.pdf), Algorithm 1).\n", "\n", "The following source code also allows to reproduce the figures of the [PEPit reference paper](https://arxiv.org/pdf/2201.04040.pdf).\n", "\n", "\n", "\n", "- [3.1 Example 1: GD with fixed step size](#3.1-Example-1-:-Gradient-descent-with-fixed-step-size-in-contraction)\n", " - [3.1.1 Worst-case guarantees as functions of the iteration count](#3.1.1-Worst-case-guarantees-as-functions-of-the-iteration-count)\n", " - [3.1.2 Worst-case guarantees as functions of the step size](#3.1.2-Worst-case-guarantees-as-functions-of-the-step-size)\n", "- [3.2 Example 2: Accelerated gradient method](#3.2-Example-2-:-An-accelerated-gradient-method-for-strongly-convex-objectives)\n", "- [3.3 Example 3: Accelerated Douglas-Rachford splitting](#3.3-Example-3-:-An-accelerated-Douglas-Rachford-splitting)\n", "- [3.4 Example 4: point-SAGA](#3.4-Example-4-:-point-SAGA)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Instead of re-implementing the methods as in Section 2 above, we directly import the corresponding examples and Python common packages (numpy, matplotlib) and PEPit.**" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "pycharm": { "is_executing": true }, "scrolled": true, "tags": [] }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import PEPit\n", "\n", "# import gradient descent in contraction from the toolbox\n", "from PEPit.examples.tutorials import wc_gradient_descent_contraction\n", "# import an accelerated gradient method for the strongly convex objectives from the toolbox \n", "from PEPit.examples.unconstrained_convex_minimization import wc_accelerated_gradient_strongly_convex\n", "# import an accelerated Douglas Rachford splitting from the toolbox\n", "from PEPit.examples.composite_convex_minimization import wc_accelerated_douglas_rachford_splitting\n", "# import point-SAGA from the toolbox\n", "from PEPit.examples.stochastic_and_randomized_convex_minimization import wc_point_saga" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the verbose parameter to 0 to only print the result without details\n", "# and to -1 to return worst-case without printing a detailed report.\n", "verbose = -1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.1 Example 1 : Gradient descent with fixed step size in contraction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We consider the following convex minimization problem:\n", "\\begin{equation}\n", "f_\\star \\triangleq \\min_x f(x),\n", "\\end{equation}\n", "where $f$ is $L$-smooth and $\\mu$-strongly convex." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The package PEPit allows to compute a worst-case guarantee for **gradient descent** with fixed step size $\\gamma$. That is, it computes the smallest possible $\\tau(n, L, \\mu, \\gamma)$ such that the following guarantee holds (for all initialization of the algorithm, and all $L$-smooth $\\mu$-strongly convex function)\n", "\\begin{equation}\n", "\\| x_n - y_n \\|^2 \\leqslant \\tau(n, L, \\mu, \\gamma) \\| x_0 - y_0 \\|^2,\n", "\\end{equation}\n", "where $x_n$ and $y_n$ are the outputs of the gradient descent with fixed step size $\\gamma$, starting respectively from $x_0$ and $y_0$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Gradient descent with fixed step size $\\gamma$ may be described as follows, for $t \\in \\{0,1, \\ldots, n-1\\}$\n", "\\begin{equation}\n", "x_{t+1} = x_t - \\gamma \\nabla f(x_t).\n", "\\end{equation}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Theoretical guarantee" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The **tight** worst-case guarantee which we compare to below, is \n", "\\begin{equation}\n", "\\| x_n - y_n \\|^2 \\leqslant \\max\\left\\{(1-L\\gamma)^2,(1-\\mu \\gamma)^2\\right\\}^n\\| x_0 - y_0 \\|^2,\n", "\\end{equation}\n", "which is tight on simple quadratic functions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3.1.1 Worst-case guarantees as functions of the iteration count" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the parameters\n", "L = 1 # smoothness parameter\n", "mu = 0.1 # strong convexity parameter\n", "gamma = 1 / L # step-size\n", "\n", "# Set a list of iteration counter to test\n", "n_list = np.array([1, 2, 4, 6, 8, 10, 15])\n", "\n", "\n", "# Compute numerical and theoretical (analytical) worst-case guarantees for each iteration count\n", "pepit_taus = list()\n", "theoretical_taus = list()\n", "for n in n_list:\n", " pepit_tau, theoretical_tau = wc_gradient_descent_contraction(mu=mu,\n", " L=L,\n", " gamma=gamma,\n", " n=n,\n", " verbose=verbose,\n", " )\n", " pepit_taus.append(pepit_tau)\n", " theoretical_taus.append(theoretical_tau)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1QElEQVR4nO3dd3xUdfb/8ddJI/TeW0IvCQmhlwQsKIpgxbKiIGLvBdf9ur+17aqrrmBBUURARFEsKCpFUQihSUIT6ZDQVQgQeklyfn/MJAZIhglkcjOT83w85pG5d2buvBNCztz7ufd8RFUxxhhjChLkdABjjDElmxUKY4wxHlmhMMYY45EVCmOMMR5ZoTDGGONRiNMBfKFGjRoaERHhdAxjjPEbKSkpe1S1Zn6PBWShiIiIIDk52ekYxhjjN0RkS0GP2aEnY4wxHgVUoRCR/iLyXkZGhtNRjDEmYARUoVDVaap6Z+XKlZ2OYowxASMgxyiM8XcnT55k+/btHDt2zOkoJsCEh4fToEEDQkNDvX6NFQpjSqDt27dTsWJFIiIiEBGn45gAoaqkp6ezfft2IiMjvX5dQB16OmdJIyE18dR1qYmu9cY44NixY1SvXt2KhClSIkL16tULvadqhQKgfhxMGfJXsUhNdC3Xj3MylSnlrEgYXziX3ys79AQQmQADx8OUIaS3HkT1NR+5liMTnE5mjDGOsz2KHJEJbG/2N6qnvM4P5a9gf+2uTicyxjHp6enExsYSGxtLnTp1qF+/PrGxsVSpUoU2bdoUa5apU6eyevXq3OV//etf/Pjjj4XeTlpaGlFRUfmu//jjj3OXk5OTefDBB89pWwDjx49n586d+T7Wu3fvYr0Y2FPOwgioQnFe11GkJlJ/48csqD+UuD+/5O+vjmL6r7uKPqQxfqB69eosX76c5cuXc/fdd/PII4/kLgcFFf2fjczMzAIfO71QPPfcc1x88cVF9t6nF4qOHTvyxhtvnPP2PBUKfxVQheKcr6Nwj0nIwPF0v2MEB/qP4WV9jQ8/mcjjU1b4JqwxfiorK4s77riDtm3bcskll3D06FEANm3aRN++fenQoQPx8fGsXbsWcP0hvvDCC2nXrh0XXXQRW7duBWDIkCHcfffddOnShSeeeCLf1y9YsIBvvvmG4cOHExsby6ZNmxgyZAiff/45AEuWLKF79+7ExMTQuXNnDh48SFpaGvHx8cTFxREXF8eCBQs8fj9PPvkk8+bNIzY2lhEjRjBnzhyuuOIKAHbv3k2fPn1o27Ytw4YNo3HjxuzZs6fAn8Pnn39OcnIyN998M7Gxsbk/m7wmTpxIbGwsUVFR/PLLLwDs3buXq666inbt2tG1a1dWrlwJwDPPPMOrr76a+9qoqCjS0tJIS0ujdevW+f47pKSkEBMTQ0xMDKNGjTq3f+TTqWrA3Tp06KCFMm+E6ua5p6w6uXGOzp/wT/1k8RZVVc3Oztbs7OzCbdeYc7R69epTlq8fveCM24cLUlVV9cjxzHwf/2zJVlVVTT90/IzHCuPpp5/WV155RVVVU1NTNTg4WJctW6aqqgMHDtSJEyeqquqFF16o69evV1XVRYsW6QUXXKCqqldccYWOHz9eVVXHjh2rV155paqqDh48WPv166eZmZkeXz948GCdMmVKbp6c5ePHj2tkZKT+8ssvqqqakZGhJ0+e1MOHD+vRo0dVVXX9+vWa8/cgNTVV27Zte8b39/PPP2u/fv3yXb7vvvv0hRdeUFXV6dOnK6C7d+/2+HPo1auXLlmyJN+fZa9evXTYsGGqqjp37tzcPPfff78+88wzqqo6e/ZsjYmJOeNnr6ratm1bTU1N9fj+0dHROneu6+/Z448/nu/3fPrvl6oqkKwF/E21wWyAng+fsSqkaS+6N+1Fd/fylJTtfLtyFy9cHUWDquWKNZ4xJUlkZCSxsbEAdOjQgbS0NA4dOsSCBQsYOHBg7vOOHz8OwMKFC/nyyy8BuOWWW3jiiSdynzNw4ECCg4M9vr4g69ato27dunTq1AmASpUqAXD48GHuv/9+li9fTnBwMOvXrz/n7zUpKYmvvvoKgL59+1K1atXcx/L7OXjjpptuAiAhIYEDBw6wf/9+kpKS+OKLLwC48MILSU9P58CBAx63k9/779+/n/3795OQ4DoR55ZbbmH69OnefrsFskLhLYWUtL1cMiKRv/dtxS1dGxMUZKcvmuLx6V3dCnysbFiwx8erlQ/z+HhhlSlTJvd+cHAwR48eJTs7mypVqrB8+fJCbat8+fIA5/z6/IwYMYLatWuzYsUKsrOzCQ8PP+9t5ie/n4M3Tj891dPpqiEhIWRnZ+cu573+4Vzf/1wE1BiFL13fqSEzH0mgY0Q1nv7mN65/dyEb/zzkdCxjSoRKlSoRGRnJlClTANch7RUrXON73bt3Z/LkyQBMmjSJ+Pj4Qr2+YsWKHDx48IzXtGzZkl27drFkyRIADh48SGZmJhkZGdStW5egoCAmTpxIVlaWx+wFbR+gR48efPbZZwDMmjWLffv2nfVn4Wl7AJ9++ing2lupXLkylStXJj4+nkmTJgEwZ84catSoQaVKlYiIiGDp0qUALF26lNTUVI/vXaVKFapUqUJSUhJA7jbPlxWKQmhQtRwTbuvE/wbGsOHPQ2zde9jpSMaUGJMmTWLs2LHExMTQtm1bvv76awDefPNNxo0bR7t27Zg4cSKvv/56oV5/44038sorr9C+fXs2bdqU+/ywsDA+/fRTHnjgAWJiYujTpw/Hjh3j3nvvZcKECcTExLB27drcvZaCtGvXjuDgYGJiYhgxYsQpjz399NPMmjWLqKgopkyZQp06dahYsaLH7eUM0hc0mB0eHk779u25++67GTt2LOAatE5JSaFdu3Y8+eSTTJgwAYBrr72WvXv30rZtW9566y1atGjh8b0Bxo0bx3333UdsbCyuoYfzJ0W1oZKkY8eO6utzlQ8cO0mlcFdTrU+XbKVtvcpE1beutaZorFmzhtatWzsdo9Q7fvw4wcHBhISEsHDhQu65554iOTzmtPx+v0QkRVU75vd8G6M4RzlF4tjJLN6YvZHfDxzjzoQmPHRRc8JDgx1OZ4wpClu3buX6668nOzubsLAwxowZ43QkRwRUoRCR/kD/Zs2aFdt7hocG8/2D8bzw/RrembOJmat+57/XtaNTRLViy2CM8Y3mzZuzbNkyp2M4LqDGKNShiYsqlwvlv9e146Pbu3AiK5u/jVnE7xnHrCutMSYgBFShcFrP5jWY9UgCowd1oE7lcKgfR+ang60rrTHGr1mhKGLlwkK4qHVtABZpW245cA+HPhrE0ZnPuYqEdaU1xvgZKxQ+1L5RFTr2vpJxJy6i7ML/saHRDWjEmeeQG2NMSWaFwofKhATzWPM/uKf8HCaXvZFqaybyzvhxTscyxivBwcG5zesGDhzIkSNHTlmfc3vppZcAVwvtli1bEhMTQ48ePVi3bh3guuAOzuzSWpxKQkfXomr5XRinNxU8V1YofMk9JhFywwSue/wd5rd/laG7noPUxNxmW8acNx+dNFG2bFmWL1/OqlWrCAsLY/To0aesz7k9+eSTua+ZNGkSK1asYPDgwQwfPhwgt3urrwuFqp7S7iKvklAo/JkVCl/asTR3TCIkOIgBV91I+E0fwo6lTEnezqCxi9mafsTplMbfFcNUvvHx8WzcuNHr5yckJOQ+v0KFCsCZ7bzzuu+++/jmm28AuPrqqxk6dCgAH3zwAU899RQAr732GlFRUURFRTFy5EjAVXxatmzJrbfeSlRUFNu2bWPIkCFERUURHR3NiBEjvGr9/d///pfo6GhiYmJyC9+YMWPo1KkTMTExXHvttbl7VFOmTCEqKoqYmJjc5ntZWVkMHz6cTp060a5dO9599918fy6ZmZncfPPNtG7dmuuuuy53m7Nnz6Z9+/ZER0czdOjQ3IaIERERuW3Nk5OT6d27N+DaUxg6dCi9e/emSZMmp8yf8Z///IcWLVrQs2fP3L2681ZQW1l/vhW6zbgDpiRv07b/mqGt/jld35+3WTOzrIW5+Ut+baA92jxX9b+RqrP/7fp6Wtv8c1G+fHlVVT158qQOGDBA3377bVVVDQoK0piYmNzb5MmTVfXU9tovv/yyXn/99ads5/R23nl98skn+vjjj6uqaqdOnbRLly6qqjpkyBCdMWOGJicna1RUlB46dEgPHjyobdq00aVLl2pqaqqKiC5cuFBVVZOTk/Xiiy/O3e6+ffvOyHa677//Xrt166aHDx9WVdX09HRVVd2zZ0/uc5566il94403VFU1KipKt2/ffsr23333XX3++edVVfXYsWPaoUMH3bx58ynvk5qaqoAmJSWpquptt92mr7zyih49elQbNGig69atU1XVW265RUeMGKGqqo0bN9bdu3erquqSJUu0V69equpqP96tWzc9duyY7t69W6tVq6YnTpzI/TkdPnxYMzIytGnTpqe0Kc9R2DbjtkfhkOs6NGDWIwl0bVKN579dzXWjF7Dhj4IbiRnjUWQCdLwdEl92fS2CM+uOHj1KbGwsHTt2pFGjRtx+++3AmYeebrjhhtzX5Hxqnz9/fqGOjcfHxzNv3jxWr15NmzZtqF27Nrt27WLhwoV0796dpKQkrr76asqXL0+FChW45pprmDdvHgCNGzema1fX1MVNmjRh8+bNPPDAA8yYMSO39bgnP/74I7fddhvlyrmmD6hWzXWx7KpVq4iPjyc6OppJkybx22+/Aa5GgUOGDGHMmDG5DQdnzZrFhx9+SGxsLF26dCE9PZ0NGzac8V4NGzakR48eAAwaNIikpCTWrVtHZGRkbh+nwYMHk5iYeMZrT9evXz/KlClDjRo1qFWrFn/88Qfz5s3j6quvply5clSqVIkBAwacdTvesCuzHVSvSlk+GNKJr5fv5Nlpv7F931Ga1/bccMyYfKUmQvJYSHjC9TUy/ryLRU5BKIxJkybRsWO+7YI8ql+/Pvv372fGjBkkJCSwd+9ePvvsMypUqHDWJnx5m/5VrVqVFStWMHPmTEaPHs1nn33GBx98cMrzFy9ezF133QW4plUtyJAhQ5g6dSoxMTGMHz+eOXPmADB69GgWL17Md999R4cOHUhJSUFVefPNN7n00ks9Zi1Mi3E4tc143hbjcGabcU/TyZ6vgNqjUIeuzD4fIsJV7esz7+8XckGrWgBM/mUrK7fvdzaY8R85YxIDx8OFT7m+5h2zKCHO1n67a9eujBw5koSEBOLj43n11VdzW5LHx8czdepUjhw5wuHDh/nqq6/ybVe+Z88esrOzufbaa/n3v/+d26I773t36dIld29owIAB9OnTh3HjxuWOF+zduxdwtS2vW7cuJ0+ePKVd96ZNm+jSpQvPPfccNWvWZNu2bVx66aW88847nDx5EoD169dz+PCZ3aW3bt3KwoULAfj444/p2bMnLVu2JC0tLXdMZ+LEifTq1QtwjVGkpKQA5E5s5ElCQgJTp07l6NGjHDx4kGnTpp31Nd4IqELhzyqUce3cHTuZxZs/beSqUfN58fs1HDvpuZe+MXlPmgBcXweOd633gZxDUjm3vGc9eeKpnTe4ikFmZibNmjUjLi6OvXv35haDuLg4hgwZQufOnenSpQvDhg2jffv2Z2xjx44d9O7dm9jYWAYNGsSLL74IeG793bdvXwYMGEDHjh2JjY3NPWT2/PPP06VLF3r06EGrVq1ynz98+HCio6OJiorKna972LBhtGnThri4OKKiorjrrrvy/YTfsmVLRo0aRevWrdm3bx/33HMP4eHhjBs3joEDBxIdHU1QUBB333034Gpz/tBDD9GxY0eCg8/ebDQuLo4bbriBmJgYLrvsstzZ/86XtRkvgTKOnuSl6Wv45JdtRFQvx0vXtqNrk+pOxzLFyNqMG18qbJtx26MogSqXDeXFa9rx8bAuZCvcMnaxq8mgMcY4IKAGswNN92Y1mPFwPIs373U1GQRW7zxAm3pnP5PDGGOKiu1RlHDlwkJyB7kXbkrn8jfm8fDkZew9fMLhZMbXAvGwsHHeufxeWaHwIx0aV+Whi5rz3a+7uPi1uXyzYqf9MQlQ4eHhpKen27+vKVKqSnp6OuHh4YV6nQ1m+6G1vx/g75+vZMX2DK5uX58RN8Q6HckUsZMnT7J9+/Yzzp035nyFh4fToEEDQkNDT1lvc2YHmFZ1KvHlvT0YNz81d+7unIJ/tgt4jH8IDQ0lMjLS6RjGAFYo/FZwkDAsvknu8mfJ25i6bCcvXRtN4+rlPbzSGGMKx8YoAkRIUBCrdmRw6chE3p+3mazswDukaIxxhhWKAHFthwbMejSBHk1r8O/v1nDNOwtYb00GjTFFIKAKhYj0F5H3MjIynI7iiLqVy/L+4I68cVN7tu09wo59Z/bdN8aYwrKzngLUoeOZuf2jJi3eQtt6lYltWMXZUMaYEsvOeiqF8jYZHD13Ezv2HWVoj0geu6QlZcPO3lzMGGNyBNShJ3Om8NBgvnswnhs7N+L9pFQuHZnIgk17nI5ljPEjVihKgUrhobxwdTSf3NEVERj8wS/syrDxC2OMd+zQUynSrWl1ZjyUwOLUdOpWLgvAqh0ZRNX3n4mejDHFz/YoSpmyYcH0bulqMrhoczpXvJnEA58sI/3QcYeTGWNKKisUpVhco6o82qcFM1btYuL/HmX+D1+e2oQuNRGSRjqWzxhTMlihKMXCQoJ48KLmfPdgPHsqtaVV0oOM+mCc68GceZjrxzma0RjjPBujMLSoXZFnH7qHmd/W4I5VT8BPu9Dksei14wjKmYfZGFNq2R6FAVxNBi8fcD1lut4BiS+zut513PRjGKl7DjsdzRjjMCsU5i+piZA8FhKeoNnWzyi/awF9Ryby7txNZGZlO53OGOMQKxTGJWdMYuB4uPApytz0Ie+XHcUdDbfz4vS1XP32Atb+fsDplMYYB3hVKESksYhc7L5fVkQq+jaWKXY7lrqKRM6YRGQCQdeP57G2hxn1tzh2ZRzl9wybbc2Y0uisTQFF5A7gTqCaqjYVkebAaFW9qDgCngtrClj0jpzIpFyY69yHjxZtoU29SsQ1qupwKmNMUfHUFNCbPYr7gB7AAQBV3QDUKrp4Rae0txn3pZwicexkFu8lbubadxbw3LTVHDmR6XAyY4yveVMojqvqiZwFEQkBSmRvclWdpqp3Vq5sLSl8xdVksCeDujTmg/muJoPzN1qTQWMCmTeFYq6I/B9QVkT6AFOAab6NZUqyiuGhPH9VFJ/e2ZWQoCCGjLMmg8YEMm/GKIKA24FLAAFmAu9rCZ7xyMYois+xk1n8krqXhBY1AWsyaIy/Oq+Ji1Q1W0Q+AhJVdV2RpzN+LTw0OLdILNyUzk1jFtGvXV2e6d+WmhXLOJzOGFMUznroSUQGAMuBGe7lWBH5xse5jB/qGFGVxy9pwQ+//UGfEXP5cul2SvCOpzHGS96MUTwNdAb2A6jqciDSd5GMvwoNDuL+C5vz/UM9aVqzAo9+toIHJy93OpYx5jx50xTwpKpmiEjedfYx0RSoWa2KTLmrGxMXbaG8e+5uVUUVgoLkLK82xpQ03hSK30Tkb0Cw+2K7B4EFvo1l/F1QkDC4e0Tu8uQl2/hq6Q5evDaapjUrOBfMGFNo3hx6egBoCxwHPgYygId8GcoEnnJhwaz9/QCXvT6Pt+dstCaDxvgRbwpFP1V9SlU7uW//BAb4OpgJLFfG1ufHx3pxYctavDxjHVe9PZ/VO63JoDH+wJtC8Q8v1xnjUa2K4Yy+pQPv3BzHHweO8+dBazJojD8ocIxCRC4DLgfqi8gbeR6qBFiDH3POLouuS++WtSgbFgzAxIVptKlXiQ6NqzmczBiTH0+D2TuBZFyHmVLyrD8IPOLLUCbw5RSJ45lZjE1KZcveIwzuFsHwS1vmnilljCkZvGnhEaqqJ4spT5GwFh7+5dDxTF6duY4JC9OoV7ksL1wTTS/31d7GmOJxvm3GO4vIDyKyXkQ2i0iqiGwu4oymFKtQJoRnBrRlyl3dKBMaxLAJS9i535oMGlNSeLOPPxbXoaYUIMu3cUxp1jGiGt8/GE9y2j7qVSkLwIpt+4lpWMXZYMaUct7sUWSo6nRV/VNV03NuPk9mSqXw0GB6Nq8BuJoMXjlqPvd8lGJnSBnjIG8Kxc8i8oqIdBORuJybz5OZUq9TRFX+3rcVs9f+SZ/XEpmSvM2aDBrjAG8Gs3/OZ7Wq6oW+iXTuRKQ/0L9Zs2Z3bNiwwek4pohs2n2IJ79YyZK0ffRrV5dRf7PPKcYUNU+D2WctFP7IznoKPNnZyqTFriaD18Q1QJNGonXbE9S0119PSk2EHUuh58OO5TTGX53XxEXuDfTD1e8pPGedqj5XNPGMObugIOGWbhG5y7MPNKDT7EEcGjCG+u37uorElCEwcLxTEY0JWN5MXDQauAFXc0ABBgKNfZzLGI+O1u/OYzxC2anD+OWDx9CcIhGZ4HQ0YwKON4PZ3VX1VmCfqj4LdANa+DaWMZ71j6nHi4/ex6LqV9F56/t8kt2HVWExTscyJiB5Uyhyzks8IiL1gJNAXd9FMsY7Nfcs5vJj37Ox9b1cdux7sjbPdTqSMQHJmzGKaSJSBXgFWIprdrsxvgxlzFnlGZNoFpnAiQ2XEvPVUGhQhQm7GtG6biU6R1qTQWOKgsdCISJBwGxV3Q98ISLfAuGqmlEc4Ywp0I6lp4xJhDXvDQPHk7kthfG/ZJO65zC3dmvME31bUcGaDBpzXry5jmKZqrYvpjxFwk6PLd2OnMjk1ZnrGbcglbqVwvnPNdFc0LKW07GMKdHOtyngbBG5VkSkiHMZ4xPlwkL4V/82fH53d8qVCeHOD5OtyaAx58GbPYqDQHlckxUdw3WKrKpqJd/HOze2R2FyHM/MIiVtH92bufpHLdu6j9iGVbDPPcac6rz2KFS1oqoGqWqYqlZyL5fYImFMXmVCgnOLxMJN6Vz99gLumpjCHwesyaAx3vLm0BMiUlVEOotIQs7N18GMKWqdIqryj8taMXf9bi5+bS6fLtlqTQaN8YI3V2YPAxKBmcCz7q/P+DaWMUUvJDiIu3o1ZfpD8bSuW4m/f/Er905a6nQsY0o8b84bfAjoBCxS1QtEpBXwgm9jGeM7TWpWYPIdXfn4l62Uc8/dnZ2tKBAcZGMXxpzOqyuzVfUYgIiUUdW1QEvfxjLGt4KChEFdG3NNXAMAPlmyletGL2DDHwcdTmZMyeNNodjuvjJ7KvCDiHwNbPFlKGOKW5WyYaTtOUy/N5J4Y/YGTmRmOx3JmBKjUPNRiEgvoDIwQ1VP+CzVebLTY8252HPoOM9OW820FTtpVacirw6MIap+ZadjGVMszuv0WBFplHMDUoHlQJ2ijWiM82pUKMObN7VnzK0d2XfkBHsPl9jPQsYUK28Gs7/D1QhQcE1cFAmswzWRkTEBp0+b2sQ3r0F4qGuge/z8VFrVrUTXJtUdTmaMM85aKFQ1Ou+yiMQB9/oskTElQE6ROJ6ZxYeLtrB592Fu7tKIJy9rRcXwUIfTGVO8vLrgLi9VXQp08UEWY0qcMiHBfPdAPMN6RvLJL1u5ZEQiP6/90+lYxhQrb3o9PZpnMQiIA6qr6qW+DHY+bDDb+MKyrfv4+xcrSd1zmLnDL6BelbJORzKmyHgazPZmjKJinvuZuMYsviiKYMb4k/aNqvLtA/GkbNmXWyRStuwlrlFVazJoApo3YxTPFkcQY/xBWEgQ3Zq6BrUXbkrnpjGLuLh1Lf59VTR1Koc7nM4Y3zhroRCRabjOesorA0gG3s25atuY0qZzZDX+2a81r85aR5/X5vJ//VpzY6eGtndhAo43g9mbgUO45skeAxwADgItsLmzTSkWHCQMi2/CzIcTiKpfmX98+Sv3fGRNBk3g8WaMoruqdsqzPE1ElqhqJxH5zVfBzoWI9Af6N2vWzOkophRpXL08H9/RhclLtlmTQROQvNmjqOC+KhtwXakNVHAvlqhLV1V1mqreWbmytV0wxUtEuKlzI66MrQ/ApF+2cs07C1j3uzUZNP7Pm0LxGJAkIj+LyBxgHvC4iJQHJvgynDH+qkb5MLbtPcIVb85j3vh/cnLj3FOfkJoISSMdyWZMYXlz1tP3ItIcaOVetS7PAPZIXwUzxp9dFl2XLk2q8+y033h7ZSWitwwi/bJ3adr5cleRmDIEBo53OqYxXvFmjAJVPQ6s8HEWYwJKtfJhvH5je2bH1OOpL8MYOfs+ODQMkse6ikSkzShs/INXhcIYc+4ual2bHk88QOi8w5D4Mssi7+RoVhu6Ox3MGC8VuteTMabwwrfPh+SxZPZ8nMi0ybz5wQf848uVHDh20uloxpyVN/NRiIgMEpF/uZcbiUhn30czJkDkGZMIufj/UfamiYwtN4otyTPo89pcflj9h9MJjfHImz2Kt4FuwE3u5YPAKJ8lMibQ7Fh6yphEmRa9KXfzRP7XI5uq5cK4d1IKO/cfdTajMR54M0bRRVXjRGQZgKruE5EwH+cyJnD0fPjMdZEJ1I1M4JvMbJZt/avJ4JK0vXRsbE0GTcnizR7FSREJxt3vSURqAjbzvDFFICwkiC5N/moyOHD0Qm6fkGx7GKZE8aZQvAF8BdQSkf8AScALPk1lTCnUObIa/7qiDQs3pXPJiEQ+WrSF7GzP88UYUxzOOnERgIi0Ai7CNW/2bFVd4+tg58MmLjL+bGv6Ef7x1Urmb0znkja1ee/WfOeSMaZIndfERSLSFEhV1VEi0hvoIyK7VHV/kaY0xgDQqHo5Prq9C1OSt1Mm1LXTn52tZKsSEmxntJvi581v3RdAlog0A94FGgIf+zSVMaWciHB9p4Z/NRlcvIWr317A6p0HHE5mSiNvCkW2qmYC1wBvqepwoK5vYxlj8qpZMZxdGUcZ8FYS/5u1juOZWU5HMqWIt2c93QTcCnzrXhfqu0jGmNP1jarDD4/0YkBsPd78aSP93khi+bb9TscypYQ3heI2XBfc/UdVU0UkEpjo21jGmNNVLR/Ga9fHMu62Thw9kcXh45lORzKlhFdnPfkbO+vJBLrjmVmUCXHNpvf+vM20qlOJns1rOJzK+LPzPeupOfAi0AYIz1mvqk2KLKExplByisTxzCw+XbKNDX8e4vqODXiqXxsql7Ujw6ZoeXPoaRzwDpAJXAB8CHzky1DGGO+UCQlm2gM9uad3U75YuoM+r81l5m+/Ox3LBBhvCkVZVZ2N6zDVFlV9Bujn21jGGG+Fhwbz976tmHpvD6pXKMP9Hy+1FiCmSHnTFPC4iAQBG0TkfmAHUMG3sYwxhRXdoDLf3N+D5dv25zYZXLQ5nS6R1azJoDkv3uxRPASUAx4EOgCDgMG+DGWMOTehwUF0iqgGuJoM3vjeIgaPW8L2fUccTmb82VkLhaouUdVDqrpdVW9T1WtVdVFxhDPGnLsukdV4pn8bktP2cumIRD5cmGZNBs058WaGux9EpEqe5aoiMtOnqYwx5y0oSBjSI5KZDycQ17gq//r6N+6cmOJ0LOOHvBmjqJG3AaB74qJavotkjClKDauV48Ohnfli6Q7CQqzJoCk8r3o9iUijnAURaYx7EiNjjH8QEa7r0IABMfUA+GjxFq56ez6/7cxwOJnxB94UiqeAJBGZKCIfAYnAP3wbyxjjS3UqhfN7xnEGvDWfV2au5dhJazJoCubtxEU1gK7uxUWqusenqc6TtfAw5uz2HznBv79bw+cp22laszyvDoyhfaOqTscyDvHUwsOrA5SqukdVvwU6lvQiYYzxTpVyYbw6MIYPh3bmeGY2R0+49yqSRkJq4qlPTk10rTelUmFHsgb4JIUxxjEJLWry02O96d7M1VRwWnodTky+9a9ikZoIU4ZA/TjnQhpHFbZQ2OWdxgSgnLOhjmdm8dbmegw+eC+HPhrEsZnPuYrEwPEQmeBoRuMcb66j6JFnsUM+64wxAaJMSDBf39+DDr2uZNyJiwhf+D82NrrBikQp580exZs5d1Q1+/R1xpjAEh4azOMt/uCe8nP4JPxGqq2ZyO5ff3A6lnFQgRfciUg3oDtQU0QezfNQJSDY18GMMQ5xj0mE3DCBgY16smnJdFpOvwsqjGdBdhu6NaluTQZLGU9XZofh6hIbAlTMs/4AcJ0vQxljHLRjae6YRAjQsms/qF2RLb8m8bcFB4lvXoMXro6mYbVyTic1xeSs11GISGNV3eK+HwRUUNUDxRHuXNl1FMYUvexsZdLiLbw0fS0KPHFpS27tFkFQkO1dBILzvY7iRRGpJCLlgVXAahEZXqQJjTElXlCQcEu3CGY92otOEdV4Ztpq7vjQPpCVBt40BWyjqgdE5GZgOvAkkAK84tNkxpgSqX6Vsoy/rRNTl+8gJOivJoNZqoRak8GA5M2/aqiIhAJXAd+o6kmsKaAxpZqIcHX7BvR3NxmcuGgLA96az6od1mQwEHlTKEYDaUB5INHdPbZEj1EYY4pXg6plST90nCtHzeel6dZkMNB4LBTuwes/VLW+ql6urpHvrcAFxZLOGOMXLmpdmx8e7cV1cQ0YPXcTl78+j5Qt+5yOZYqIx0LhvsDuidPWqapm+jRVHiLSRETGisjnxfWexpjCq1w2lP9e145Jw7qQma0cz7S9ikDhzaGnH0XkcRFpKCLVcm7ebFxEPhCRP0Vk1Wnr+4rIOhHZKCJPetqGqm5W1du9eT9jjPN6NKvB7Md60b2pq8ngu3M38fO6Px1OZc6HN2c93eD+el+edQo08eK144G3gA9zVohIMDAK6ANsB5aIyDe4rvZ+8bTXD1VV+w0zxs/knP10IjObr5btYO3vB7mmfX3+3xVtqFo+zOF0prDOWihUNfJcN66qiSIScdrqzsBGVd0MICKTgStV9UXginN9LxG5E7gToFGjRmd5tjGmOISFBPH1/T0Y9dNG3p6zibnrd/PslW3pF13X2oD4EW+6x4aKyIMi8rn7dr/7dNlzVR/Ylmd5u3tdQe9fXURGA+1FpMApWFX1PVXtqKoda9aseR7xjDFFqUxIMI9e0pJpD/SkXpWyPPrpCnZmHHM6likEbw49vQOEAm+7l29xrxvmq1B5qWo6cHdxvJcxxnda163EV/d2Z+WODOpXKQtA0oY99GhmTQZLOm8KRSdVjcmz/JOIrDiP99wBNMyz3MC9zhgT4EKCg4hzz8u9cFM6g8YupnvT6rx0TTsaVbcmgyWVN2c9ZYlI05wFEWkCnM95b0uA5iISKSJhwI3AN+exPWOMH+oSWY3/XB3Fyu0ZXDoykbFJqWRlW9OHksibQjEc+FlE5ojIXOAn4DFvNi4inwALgZYisl1Ebndfg3E/MBNYA3ymqr+dW3xjjL8KChJu7tKYHx5NoFvT6jz/rTUZLKkKbDMuIg8DC4CluE5dbel+aJ2qHi+WdOfI2owb419UlW9W7CQkKIh+7eqSla1kZWvuXN7G9861zXgDYCTwJzAL1yGiRrh6PpVIItJfRN7LyLDGZMb4ExHhytj69GtXF4CJC9MY8FYSK7btdzaYATwUClV9XFW7A3WAfwB7gduAVSKyupjyFYqqTlPVOytXrux0FGPMeWhUvRz7jpzg6rfn8+L3azh6wtqBOMmb/bqyuObJruy+7QQW+zKUMaZ0u7CVq8ngDZ0a8W7iZi57PZGULXudjlVqFXh6rIi8B7QFDuIqDAuA11TVWkIaY3yuUngoL14TTf+Yuvzfl79yPDPb6UillqfrKBoBZYANuK5z2A7sL4ZMxhiTq3vTGvzwaK/c/lHvzNlEi9oVuKh1bYeTlR4FFgpV7SuuyyXbAt1xnRIbJSJ7gYWq+nQxZTTGlHJ5mwx+s2Ina3YdYEBMPZ7u34bqFco4nC7wnW0+ClXVVcD3uObLng80BR4qhmzGGHOKsJAgvr6vB49c3ILpq3bRZ0QiXy/fQUGn+ZuiUWChcDcCnCwiW4G5uDq7rgWuAbyaj8IYY4paWEgQD13cnG8fiKdhtXIMn7LSmgz6mKcxighgCvCIqu4qnjjnR0T6A/2bNWvmdBRjjI+1rFORL+/pzq95mgwmrt9Nz2Y1CAqyJoNFqcArs/2ZXZltTOmzcFM6N41ZRNcm1XjpmnZE1Cix1waXSOd6ZbYxxvgNV4GI5rcdB+j7eiJjEjdbk8EiYoXCGBMQRIQbOzfih0d70bNZDf7z/Rpun7DE6VgBwZv5KIwxxm/UqRzOmFs78u3KXQS7xyqyspXM7GzKhAQ7nM4/2R6FMSbgiAj9Y+pxebSryeCHC9Po/2YSy7ZaY4lzYYXCGBPwImuU5+CxTK55ZwHPf7uaIycynY7kV6xQGGMCXu+WtZj1SAI3d2nE2KRU+o6cx5I0azLoLSsUxphSoWJ4KP++KprJd3YlJEjsjKhCCKjrKPJccHfHhg0bnI5jjCmhMrOyCXH3j3p7zkaa16pInzalu8lgqbmOwiYuMsZ4IyRPk8HvVu7ijg+Tuf/jpew5VKJneXZMQBUKY4wpjLCQIKbe14PH+rRg1m9/0Oe1uUxdZk0GT2eFwhhTqoUGB/HARc357sGeRNQozxNfWJPB09kFd8YYAzSvXZHP7+7Obzv/ajL487o/6dW8ZqlvMmh7FMYY4xYcJLRrUAVwNRm8bdwSbhyziM27DzkbzGFWKIwxJh9dm1Tj5evasXbXAS57fR6j524iM6t0zttthcIYY/IhIlzfsSE/PtqL3i1r8tL0tQydUDqnL7AxCmOM8aBWpXBGD+rA9FW/kzNUUdqaDNoehTHGnIWIcHl0XfpGuZoMjl+QRr83kkjZUjqaDFqhMMaYQmpeqwJHT2Rx3egFPDvtNw4fD+wmgwFVKESkv4i8l5GR4XQUY0wAS2hRk5mPJHBL18aMm5/GpSMTWbw53elYPhNQhcJaeBhjikuFMiE8d2UUn93VjTIhAfWn9Aw2mG2MMeehc2Q1Zj3SK3c2vbd+2kCzWhXpG1XH4WRFJ7DLoDHGFIOcInEiM5sZv/3O3R+lcO+kFP48GBitQKxQGGNMEQkLCeKre3sw/NKW/LjmT/q8lsjnKdv9vsmgFQpjjClCocFB3HdBM75/MJ7mtSrwf1/9yo79R52OdV5sjMIYY3ygWa0KfHZXN1bvOkCDquUA+GntH/RuUcvvmgzaHoUxxvhIUJAQVd91FuaizekMHZ/MDe8tZJOfNRm0QmGMMcWgS2Q1Xh0Yw/o/DnHZ6/N4e85GTvpJk0ErFMYYUwxEhOs6NOCHRxO4qFUtXp6xjqHjlzgdyys2RmGMMcWoVsVw3hnUgem/7kLENVaRla2czMomPLRkNhm0PQpjjHHAZdF1cy/KG78gjcvfmEdy2l6HU+UvoAqF9XoyxvijlrUrcvxkNgPfXcjTX6/iUAlrMij+fiFIfjp27KjJyaVzghFjjH86fDyTV2auY8LCNOpVLsurA2Po1rR6sb2/iKSoasf8HguoPQpjjPFX5cuE8MyAtnx+dzfKhQXntgUpCWww2xhjSpAOjasx4+GE3ELx5uwNNK1Vgcuj6zqWyfYojDGmhMnbZPDHNX9w76Sl3D0xhT8PONNk0AqFMcaUUGEhQXxxT3f+3rcVP637k4tfm8tnyduKvcmgFQpjjCnBQoKDuKd3U2Y8FE+rOpX459RVxd5k0MYojDHGDzSpWYHJd3Y9pcngj6v/4IJWtXw+8G17FMYY4ydObzI47MNkJr32KDuWzTj1iamJkDSy6N63yLZkjDGm2HSJrMZr18eQeLghZacO46svP3E1GUxNhClDoH5ckb2XHXoyxhg/JCJcE9eA+Ob38eGnlbllxXBWZqTQYfeXMHA8RCYU2XvZHoUxxvixmhXL8PCw2zkQdSsdtoyBjrcXaZEAKxTGGOP/UhOJTJ0MCU9A8ljX4aciZIXCGGP8Wc6YxMDxcOFTrq9ThhRpsbBCYYwx/mzH0lPHJCITXMs7lhbZWwRU91gR6Q/0b9as2R0bNmxwOo4xxviNUtM9VlWnqeqdlStXdjqKMcYEjIAqFMYYY4qeFQpjjDEeWaEwxhjjkRUKY4wxHgXUWU85RGQ3sMXpHKepAexxOoSX/Ckr+Fdef8oK/pXXn7JCycvbWFVr5vdAQBaKkkhEkgs69ayk8aes4F95/Skr+Fdef8oK/pXXDj0ZY4zxyAqFMcYYj6xQFJ/3nA5QCP6UFfwrrz9lBf/K609ZwY/y2hiFMcYYj2yPwhhjjEdWKIwxxnhkhcKHRKShiPwsIqtF5DcRecjpTGcjIsEiskxEvnU6y9mISBUR+VxE1orIGhHp5nQmT0TkEffvwSoR+UREwp3OlENEPhCRP0VkVZ511UTkBxHZ4P5a1cmMeRWQ9xX378JKEflKRKo4GDFXflnzPPaYiKiI1HAim7esUPhWJvCYqrYBugL3iUgbhzOdzUPAGqdDeOl1YIaqtgJiKMG5RaQ+8CDQUVWjgGDgRmdTnWI80Pe0dU8Cs1W1OTDbvVxSjOfMvD8AUaraDlgP/KO4QxVgPGdmRUQaApcAW4s7UGFZofAhVd2lqkvd9w/i+kNW39lUBRORBkA/4H2ns5yNiFQGEoCxAKp6QlX3Oxrq7EKAsiISApQDdjqcJ5eqJgJ7T1t9JTDBfX8CcFVxZvIkv7yqOktVM92Li4AGxR4sHwX8bAFGAE8AJf6MIisUxUREIoD2wGKHo3gyEtcvbrbDObwRCewGxrkPlb0vIuWdDlUQVd0BvIrr0+MuIENVZzmb6qxqq+ou9/3fgdpOhimkocB0p0MURESuBHao6gqns3jDCkUxEJEKwBfAw6p6wOk8+RGRK4A/VTXF6SxeCgHigHdUtT1wmJJ1aOQU7uP7V+IqcPWA8iIyyNlU3lPXefQl/pMvgIg8heuw7ySns+RHRMoB/wf8y+ks3rJC4WMiEoqrSExS1S+dzuNBD2CAiKQBk4ELReQjZyN5tB3Yrqo5e2if4yocJdXFQKqq7lbVk8CXQHeHM53NHyJSF8D99U+H85yViAwBrgBu1pJ7kVhTXB8YVrj/vzUAlopIHUdTeWCFwodERHAdQ1+jqq85nccTVf2HqjZQ1Qhcg6w/qWqJ/cSrqr8D20SkpXvVRcBqByOdzVagq4iUc/9eXEQJHnx3+wYY7L4/GPjawSxnJSJ9cR06HaCqR5zOUxBV/VVVa6lqhPv/23Ygzv07XSJZofCtHsAtuD6dL3ffLnc6VAB5AJgkIiuBWOAFZ+MUzL3n8zmwFPgV1/+9EtPCQUQ+ARYCLUVku4jcDrwE9BGRDbj2iF5yMmNeBeR9C6gI/OD+vzba0ZBuBWT1K9bCwxhjjEe2R2GMMcYjKxTGGGM8skJhjDHGIysUxhhjPLJCYYwxxiMrFMavicghL57zsPtq2KJ6z6vyNncUkedE5OKi2r6vuLvt3ut0DuN/rFCY0uBhXE34vCYiwR4evgrILRSq+i9V/fGckhWvKoAVClNoVihMQBCR3iIyJ8/8FJPE5UFcvZV+FpGf3c+9REQWishSEZni7sWFiKSJyH9FZCkwUETuEJElIrJCRL5wX1XdHRgAvOK+qKupiIwXkevc27jI3aTwV/c8BGXybPtZ93v+KiKt8vkegkXkVfd8FStF5AEvtlnDfb+jiMxx33/G/bw5IrLZ/TMA1wVzTd25XzntvSPENafHGHHNmTFLRMoW7b+S8VdWKEwgaY9r76EN0ATooapv4GrnfYGqXuD+w/pP4GJVjQOSgUfzbCNdVeNUdTLwpap2UtWcuS5uV9UFuFpbDFfVWFXdlPNCcU1ENB64QVWjcTUuvCfPtve43/Md4PF88t8JRACx7jkVJnmxzYK0Ai4FOgNPu3uOPQlscucens9rmgOjVLUtsB+41ov3MaWAFQoTSH5R1e2qmg0sx/VH93RdcRWS+SKyHFcPo8Z5Hv80z/0oEZknIr8CNwNtz/L+LXE1/lvvXp6Aa86MHDlNIVMKyHYx8G7OnAqquteLbRbkO1U9rqp7cDXz86ZFeKqqLj9LRlMKhTgdwJgidDzP/Szy//0W4AdVvamAbRzOc388cJWqrnB3Je1dRPkKylZYmfz1Ye/0aVW9+Vmc7vTX2KEnA9gehSkdDuJqFgeumc96iEgzABEpLyItCnhdRWCX+7DNzQVsL691QETOtnE1hJxbiJw/AHeJawY8RKTaWbaZBnRw3/fmMFFBuY3xyAqFKQ3eA2aIyM+quhsYAnzi7jq7ENfx/Pz8P1wzEs4H1uZZPxkY7h5gbpqzUlWPAbcBU9yHq7KBwnQwfR9XO/KVIrIC+NtZtvks8LqIJOPaA/BIVdNxHXJbdfpgtjGeWPdYY4wxHtkehTHGGI+sUBhjjPHICoUxxhiPrFAYY4zxyAqFMcYYj6xQGGOM8cgKhTHGGI/+P2ZIWgU5SnwWAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot theoretical and PEPit (numerical) worst-case performance bounds as functions of the iteration count\n", "\n", "plt.plot(n_list, theoretical_taus, '--', label='Theoretical tight bound')\n", "plt.plot(n_list, pepit_taus, 'x', label='PEPit worst-case bound')\n", "\n", "plt.semilogy()\n", "plt.legend()\n", "plt.xlabel('Interation count n')\n", "plt.ylabel('Worst-case guarantee')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3.1.2 Worst-case guarantees as functions of the step size" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the parameters\n", "n = 1 # iteration counter\n", "L = 1 # smoothness parameter\n", "mu = 0.1 # strong convexity parameter\n", "\n", "# Set a list of step-sizes to test\n", "gammas = np.linspace(0, 2 / L, 41)\n", "\n", "\n", "# Compute numerical and theoretical (analytical) worst-case guarantees for the each step-size\n", "pepit_taus = list()\n", "theoretical_taus = list()\n", "for gamma in gammas:\n", " pepit_tau, theoretical_tau = wc_gradient_descent_contraction(mu=mu,\n", " L=L,\n", " gamma=gamma,\n", " n=n,\n", " verbose=verbose,\n", " )\n", " pepit_taus.append(pepit_tau)\n", " theoretical_taus.append(theoretical_tau)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABHd0lEQVR4nO3dd3yUVfb48c9JSKEXAUVqqEoNEEBaQBTFsrAoVUAC2JG1rPpz17VhWb/KiooionTpqCwqIohoQFAJVYrUUIKsQkINLeX8/pgnOAmTMECmJJz36zWvzDz15Mlk7tzn3nuuqCrGGGNMTiGBDsAYY0xwsgLCGGOMR1ZAGGOM8cgKCGOMMR5ZAWGMMcajIoEOIL+UL19ea9SoEegwjDGmQFm1atVBVa3gaV2hKSBq1KhBQkJCoMMwxpgCRUR257bObjEZY4zxyAoIY4wxHlkBYYwxxiMrIIwxxnhkBYQxxhiPfFZAiMh4EflDRDbksl5E5B0R2S4i60Wkmdu6gSKyzXkM9FWMLHsLEuOzL0uMdy03xphg5ofPL1/WICYCXfJYfwtQx3ncB7wPICLlgOeBVkBL4HkRKeuTCCs3g9lxf17kxHjX68rN8trLGGMCzw+fXz4bB6Gq8SJSI49NugGT1ZVv/EcRKSMilYCOwCJVTQEQkUW4Cprp+R5kVCz0nIjOjuN4o7sp+ctk6DnRtdwYY4KZ8/nF7DiIGQIJ4/L98yuQbRCVgb1ur5OcZbktP4eI3CciCSKScODAgYuLIiqWn8r9lZI/jWR3zb5WOBhjCo6oWBYWux3iX3cVEvn8+VWgG6lVdayqxqhqTIUKHkeKn19iPC0OfsbUiN6U+GUScz+djk2iZIwpEJzPr4XlB7pqEDnbJC5RIAuIfUBVt9dVnGW5Lc9/zj270N6T6P730UyrNpz2657kzQ8/4tipNJ+c0hhj8kViPDo7jofODGPTNQ//ebspHwuJQBYQ84C7nd5M1wFHVHU/8DVwk4iUdRqnb3KW5b99q8/esysWXoSHBw/ix2b/QZNWk/rtf6yHkzEmeO1bzf7O77MiswFVyxb7s01i3+p8O4XPGqlFZDquBufyIpKEq2dSGICqjgHmA7cC24ETwCBnXYqIvASsdA41PKvBOt+1ezRnzNzWrTetb+xOuT9+hNlx7LlhNNWad/mzh0DPiT4JxRhjLki7R9m29QDwM1XLFXMti4rN13YIX/Zi6nue9QoMzWXdeGC8L+LyRrni4RAVy6qWI6kx7x6Wr+5J60P/RayHkzEmiJSMLMJtjSpRo3wxnxy/0KT79oUGbW/nh02LuWHfeOaW6keHitfhmwEZxhhz4ZpVK0uzfr77VCrQvZh8LTLpB244/gUbat9P7JF5PPf2+2zYdyTQYRljDABn0jN9enwrIHLj1ubQsP/rpNw6luFnRnBk0+JAR2aMMQDc+f5yHpq6ymfHtwIiN249nABqt7qVondNpm3RPQAkff4qadu/z76P9XIyxvjR3kMnKFss3GfHtwIiN+0ePadBOrLu9dDuUf535BT/+DmcE1P7k7LhG9dKy+NkjPGjY6fSOHwi7c8eTD5gjdQX4arSkfTp2Y9H5sDIOYNI2jqQKtunWR4nY4zf7E05CeAaA+EjVoO4SLc1rsQ/h97P52FdqLJ+FGuuvNMKB2OM3+w9dAKAquWK+uwcVkBcgron1tC/yGK+KNOfentn5XseFGOMyU2VskW5p10U1a8o7rNz2C2mi+W0OYT0mshtNdqTubMXzI5je4d3CakZS80KJQIdoTGmEGtwdWkaXF3ap+ewGsTFcuvlJCKE1upA5p0T+G7J18x99ylWLpmbfXvr4WSMyUf7j5zkVFqGT89hBcTF8tDLKaRWB2554DWSSzeg5ncPM2Pmx2RkqvVwMsbku7vH/cyw6Wt8eg67xZTPKpcpyrPDHmDK9BLcselp5o1czl/TFyC9JlojtjEmX6gqSYdOElv3IufB8ZLVIHwgMiyUe+8eyP/q9KP7sakQM9gKB2NMvjl4/Awn0zKoWtZ3PZjACgjfSYynwW+z0dgnkVXjOfjLImau3GOz1RljLtmfXVx9NwYC7BaTb7jlcRInP3uxqQOYe2IoPyfezCvdGxIZFhroKI0xBdTeFP8UEFaD8IUceZyIiiWi72QerHOUT1YnMe3Nx/l93aLs+1gvJ2OMlxpVLs2zt9f36ShqACkstzxiYmI0ISEh0GGc15Jf/+DjGVN4g5Gk3DKW2q1uzT5bnbVVGGP8SERWqWqMp3V2i8nPrr+mIrWGPcjkz8rwyHcPQ+oQSBhnhYMxxmvrkw5Trng4VXxcg7BbTAFQ7YpiPHrPEKTFEIh/ne9LdSWl4nWBDssYU0AMnbaa/1uwxefnsQIiUBLjIWEcSY2G0Wj/HJ5/633W7T0c6KiMMUEuPSOT3w6f8nkXV7ACIjDc2hyq3Pkyh24by/C0EYz44COm/rTbusIaY3K1/8gpMjLV5z2YwMcFhIh0EZEtIrJdRJ72sL66iCwWkfUi8p2IVHFblyEia53HPF/G6Xc5ejnVankrRXpP4vby+3nmsw0sm/TsuZlhrZeTMYY/x0BU80MB4bNGahEJBd4DOgNJwEoRmaeqm9w2GwFMVtVJItIJ+DcwwFl3UlWjfRVfQLV79JxFJa/tRM9613Pw+x3UKRuWvVeTey8nY8xlLckPEwVl8WUvppbAdlXdCSAiM4BugHsBUR943Hm+BJjrw3iCXkiIMPT62kBttNQEUj/uz+91+1Fr90zr5WSMASC2bgXe79eMSmUifX4uX95iqgzsdXud5Cxztw64w3neHSgpIlc4ryNFJEFEfhSRv3o6gYjc52yTcODAgXwMPfCOXd2GL8Jvodbm0Swr2430au0CHZIxJghcVTqSWxpVIizU903IgW6kfgLoICJrgA7APiArwXl1Z/DGXcBbIlIr586qOlZVY1Q1pkIF32Y19LdS+1fQWxbx7ZVxXJs0m1dHj+WPY6cCHZYxJsC+2fQ7G/Yd8cu5fFlA7AOqur2u4iw7S1V/U9U7VLUp8Iyz7LDzc5/zcyfwHdDUh7EGF6fNQXpOpNODb7Oxzds8fPBlRoz5iMxM6+FkzOXsn5/9wqTlu/xyLl8WECuBOiISJSLhQB8gW28kESkvIlkx/AMY7ywvKyIRWdsAbcnedlG45ejlFHvznRzv+hFD6x0lZPnbZO74PntXWOvhZMxl4VRaBn8cO+2XLq7gwwJCVdOBh4Gvgc3ALFXdKCLDRaSrs1lHYIuIbAWuBF5xll8LJIjIOlyN16/l6P1UuHmYra5a8y5U7/pPqNyM0zPuZuSH4zh6Ks1mqzPmMpJ0Ns237wfJgY9zManqfGB+jmXPuT2fA8zxsN9yoJEvYyuwomJZ0ugNBq56nM/+8xP9Qr+hSO9J1sPJmMvAXj92cYXAN1Kbi3Br116cahLHwLRZfHCiI3NSagY6JGOMH/hroqAsVkAURInxVN4+jdTrHmdAkcV89ul0Eg+mBjoqY4yP/bVpZeYObUuFEhF+OZ+l+y5o3EZVF4+KJaNORybOiiPsWCsoH8uRk2mULhoW6CiNMT5QKjKM6Kpl/HY+q0EUNDl6OIXW6kBYn0mwbzWLN//O+P97lITv/pt9H+vlZEyh8PGPu/lh+0G/nc8KiILGQw8nomKh3aPUvbIkB0rVJ2rJUKbN+Jj0jEzr5WRMIfL6gl9ZsOF/fjuf3WIqRKqWK8Zzwx7g4xkl6L75aeaO/IE7Mr4mpNdE6+VkTAF35GQaR0+l+62LK1gNotCJDAvlngEDOVCvHz2OT2Nz5Z5WOBhTCOxNcXow+amLK1gBUTglxnNN0myOtXqM+r/NhsR4tvzvmKXpMKYAS/JzF1ewW0yFj1svp5JRsXBNJzJmDeTfqUMhKpaRvaIpWzw80FEaYy5Q0iH/DpIDq0EUPjl6OREVS0jPifztmmMs357MtJF/Z9tPX2bfx3o5GRP0BreN4ud/3kDpYv7rxu5VAeFMDXqj87yoiJT0bVjmonno5SQ1O9DsrheZ82Brfg2pzRXz72fB57NcK62XkzEFQkiIULGU7ycJcnfeW0wici9wH1AOqIUrbfcY4AbfhmbyW+MqZXj50Yf4YFIxhq59Eopvg4RxNludMQXAG1//SoOrS3Nro0p+O6c3NYihuNJtHwVQ1W1ARV8GZXyndLEwnrj/HiLb3Avxr/PHNf3ZFBEd6LCMMXlQVT5amsiaPYf8el5vCojTqnom64WIFAGsO0wBFrJ7KaGrxkPsU0Sumchr749l5so92eeYMMYEjQPHTnM6PdOvPZjAu15M34vIP4GiItIZeAj43LdhGZ9x6+VEVCx61XW8O2cQ93+WSbGVB+l84y1E1r0++/b7VrvaNowxAeHvLK5ZvKlBPA0cAH4B7sc1v8O/fBmU8aEcvZxK17+B4v2m8FDdo0xLKs+p6XeTsuEb17bWgG1MUPD3PBBZzluDUNVMEfkYiFfVLX6IyfiSh5pAaK0OtK/VAbYdYMKScjw6/z74Y4g1YBsTJI6cTCO8SAhVyvovzQaAnO++szM96BtAuKpGiUg0MFxVu+a5o5/FxMRoQkJCoMMoHL59BeJfZ1nlwcTEjSAyLDTQERlz2cvMVEJCJN+PKyKrVDXG0zpvbjE9D7QEDgOo6logKr+CM0EmMR4SxrGxzgNcmzSbF94ewy6bjMiYgPNF4XDec3qxTZqqHsmxzLq7FEZuDdgN+v0fu69/j6eO/5sXR43hy/X7Ax2dMZetx2auZdbKvX4/rzcFxEYRuQsIFZE6IjIKWO7juEwg5GjAbtaxGxl3TqBjib0MnbaatdNfcBUi7ixNhzG+sewtSIwnLSOT/67d5+rJ5Of/N28KiGFAA+A0MA04AjzizcFFpIuIbBGR7SLytIf11UVksYisF5HvRKSK27qBIrLNeQz07tcxl8RDmo4KjTrT97E3efTGOtSKjnXVMLIKCevlZIzvVG4Gs+NI2bCYTIVmGRv8/v/mTSN1T1Wdfb5lHvYLBbYCnYEkYCXQV1U3uW0zG/hCVSeJSCdgkKoOEJFyQAIQg+t21iqguarmOozQGqn9I33H95ycOoA/6vWj1u6Z1svJGF9KjCdtxkBGp3bgoeLfu6YXzuf/t0ttpP6Hl8tyaglsV9WdzkjsGUC3HNvUB751ni9xW38zsEhVU5xCYRHQxYtzGh87VPE6voy4hVqbRxNfuitnqrYLdEjGFF5RsWyp2pNHinzGiSYD/f5lLNcCQkRucdobKovIO26PiUC6F8euDLi3qiQ5y9ytA+5wnncHSorIFV7uawKgwsGf6C2L+O6qOBr8Nofho8acncjEGJPPEuOpvWcWk8J6UWrD5HPbAH0srxrEb7hu85zCdYsn6zEP1zf8/PAE0EFE1gAdgH1Ahrc7i8h9IpIgIgkHDhzIp5BMrpw2B+k5kY4PvM222FE8fvhVPpg00fI4GZPfnP+3yL6TGfjMh0jPidnbAP3AmzaIMFVNu+ADi7QGXlDVm53X/wBQ1X/nsn0J4FdVrSIifYGOqnq/s+4D4DtVnZ7b+awNwg+WveVqIHOr5v5v3UJC96+hQokIzlwZjdSMJSzU+d5heZyMuXge/t988T+VVxuENwVEW+AFoDqu1BwCqKrWPM9+RXA1Ut+Aq2awErhLVTe6bVMeSHHSebwCZKjqc04j9Sogq7l+Na5G6pTczmcFRIAlxnP84/78X8mnuX9gHFUOJ2RLCmiMuXCqSrf3fqBXTFX6X1fdJ+fIq4DwJpvrOOAxXB/YXt/+UdV0EXkY+BoIBcar6kYRGQ4kqOo8oCPwbxFRIB7X3BOoaoqIvISrUAFXao9cCwcTBKJi2dDmbR6LH8bstxMYFPEt4X0mW+FgzCU4dCKN9UlH6BYdmCZYbwqII6r61cUcXFXn48r+6r7sObfnc4A5uew7Hhh/Mec1gXHdDd05fGot9698i7dTu3NsY3meqppJeBGb+tyYi7EnxdUBpJqf03xn8eY/d4mIvCEirUWkWdbD55GZgicxnjIbp5De7gnuiVzCzpVf8fvRU4GOypgCa3eyKw9a9SsCU0B4U4No5fx0v0elQKf8D8cUWG55nIpExVKkVgc+mhVHyJEWaNn2/Pbla1Ru0NbnDW7GFCZ7kgNbg/BmPojrz7eNMTnzOBEVS0ivibBvNfOO1GLG8lA+WjuA8D6TCavdIfvMdsYYjyqWiuCm+lcGLOX+eXsxAYjIbbjyMUVmLVPV4T6M64JZL6bgdTo9g3/P/5UtP37JmIhREDOE0hsnWw8nY4LAJaXaEJExQG9cSfsE6Imry6sxXokoEsoLXRsw8K4BTNfOlF45kp01+ljhYMx5BHoAqjeN1G1U9W7gkKq+CLQG6vo2LFMYdSm+jXsjlzCrWF+q7Jjm97QBxhQkp9IyaPj810z9aXfAYvCmkTqrG8oJEbkaSAYq+S4kUyg5bQ6hvSfRo3p7QnYvhdlxLGn8BtWb30zNCiUCHaExQWVPyglSz2RQIsKbj2nf8KYG8bmIlME1L/VqYBeueSGM8Z5bI3ZIiEBULCe6fcQvP3/LX0YtY+Ps4TYZkTFudjs9mKpfUTxgMeRZQIhICLBYVQ+r6ie42h6ucR/sZoxXPExGVKxeJ3r8bQT1ry7Fy2siOf5xf05v/c610iYjMpe5s2MgAtTFFc5TQKhqJvCe2+vTHuanNuaiXV2mKNPvvY5mHbpy78mHOTV9ABmLX7Y8TuaytyflBCUji1CmWFjAYvDm5tZiEbkT+FQD3aRuCqUioSE8efM1xEddwe6fDtJ46RsQ+5QVDuayFl21DMUjiiAiAYvBmwLifuBxIF1ETvFnNtdSPo3MXHZiwzbD/jkQ+xRnfvqQD3dVYmC/uwPaSGdMoNzRrEqgQ/BqJHVJfwRiLnPuI6ujYvnmeB36rHqcZ0ceZ8iAgTSsXDrQERrjN5mZyrFT6ZQO4O0l8K4XEyJSVkRaikhs1sPXgZnLTI5UHbd27cUfN4+hVtpW5o95mgWfz8o+aMh6OJlC7LcjJ2kyfCGzEvaef2Mf8mYk9T245mr4GnjR+fmCb8Mylx0PvZyubXM7dz3+JlzdjBYJj7Nu6eeuFdbDyRRyWUn6qpQpGtA4vLm5+wjQAvhRVa8XkWuAV30bljEu5YqH8+QD97Dm+4o0/fFRSP+FzJXjXIkArRHbFFK7s+aBCFCa7yxejaRW1VMigohEqOqvIlLP55EZ4xARmnXsBpkbIP51xnInGbuu5sHq6hp0Z0whszv5BGGhQqXSga1BeNMGkeSMpJ4LLBKR/wKBSw5iLk+J8ZAwjtNt/k6/0G9YtuhT7h7/M38cswmJTOGzJyWVqmWLERrgL0De9GLq7jx9QUSWAKWBBT6Nyhh3bj2cIqJiCa/dgQkzBnL/7hBuffsosxr+TM0m7W0yIlNodG1SmY510wIdhleN1NWyHkAisBa4yteBGXNWjh5OUrMDkX0nM6JtBuWKh/PdsSquAiQrl5M1YpsCrkvDq+jVomqgwzj/hEEi8guuKUYF14RBUcAWVW3g+/C8ZxMGXZ5OnslABCKTfiB95kBSGw+k9AabjMgUXKfSMth5IJWaFYr7ZSa5S5owSFUbqWpj52cdoCWwIr+DNOZiFA0PJTIsFK3RnjlyE6V/Hsm2ar2tcDAF1ub9R7n1naUs23Yw0KF4N1DOnaquBlp5s62IdBGRLSKyXUSe9rC+mogsEZE1IrJeRG51ltcQkZMistZ5jLnQOM3lRXYtpacuZGaxvpTbPIUPJ0/i5JmMQIdlzAXbEyRdXMGLRmoRedztZQjQDPjNi/1CcWWC7QwkAStFZJ6qbnLb7F/ALFV9X0TqA/OBGs66Haoa7c0vYS5zbpMR3VGtHbNnt+OOzU/z3NtHeeL+e7myVOT5j2FMkMgaJFctgGm+s3hTgyjp9ogAvgS6ebFfS2C7qu5U1TPADA/7KZCV9K80XhQ8xpzDrRE7LDSEu/r0Z9+No2kTuYfy6963iYhMgbI75QRXlorwS/vD+XjTzfXFizx2ZcA9kUgS596aegFYKCLDgOLAjW7rokRkDXAU+JeqLs15AhG5D7gPoFq1ahcZpinwPHRlbdy+K43bd4XEeDJnxTGu0nP06tGP0r+v+DMpoDFBaE/yiaCoPYB3t5g+x/VN390RIAH4QFUvZaRSX2Ciqv5HRFoDU0SkIbAfqKaqySLSHJgrIg1U9aj7zqo6FhgLrl5MlxCHKayiYklo8R/u/H4Yn/5nGf2LLCaszyRrxDZB67HOdcnIDI6PM29uMe0EjgMfOo+jwDGgrvM6N/sA9468VZxl7oYAswBUdQWubrTlnZnrkp3lq4AdzvmMuWAtO3XnTNNBDMqYzZgTHRm1s1LQ/AMak1PrWlfQrk75QIcBeFdAtFHVu1T1c+fRH2ihqkNxNVjnZiVQR0SiRCQc6APMy7HNHuAGABG5FlcBcUBEKjiN3IhITaAOroLKmAuXGM9VW6dyus3fGRS+mOWLP+Ptb7YGOipjzpGSeoZvf/2dIycDP4oavCsgSjijqAFX11SghPPyTG47qWo68DCu9OCbcfVW2igiw0Wkq7PZ34F7RWQdMB2Ic6Y1jQXWi8haYA7wgKqmXNivZgzZ03Tc9BzF+01hYonR3FPVVZlNix9pjdgmaKzafYjBExPYeeB4oEMBvMvm+ndgmYjswDWaOgp4SESKA5Py2lFV5+Pquuq+7Dm355uAth72+wT4xIvYjMmbhzQdEX0nE7FvNWkZHRm+OoKn4wcQ1nsS4XU6Zp/Zzhg/252cCkD1K4oHOBIXb3oxzReROsA1zqItbg3Tb/kqMGPyhadkfVGxEBWLpmcSXrsj9yw/zvvTBnCs2SCu2PyxpekwAbMn5QQlI4pQNsBTjWbxajZ4VT0NrPNxLMb4VXiREJ69vT5Lapdn9swt3LvqbTbUvp8GNdpjs0yYQNiTcoJqVxRDJDjegRecasOYwub6iF8ZErmEz0r2o8qO6aRt/z7QIZnL1J7kE1QPghQbWbyqQRhTaDltDiG9JtKtenuObLqD8M8Gc/qv49kQ0YTm1csGOkJzGRkzoHlQ1V69GSgnQD+gpqoOd3oxXaWqP/s8OmN8za0ROwQo2/BGKD6RH5d8zaBtqUysu4K2sZ0JrdXhz31sMiLjI3WvLBnoELLx5hbTaKA1rlHP4Bok957PIjLGn9o9em6DdFQszfu9SPemVXh/WymOf9yfA+sXudbZZETGRxIPpjLlx90cPpHr6AG/8+YWUytVbebkRUJVDzkD34wptEpEFOE/vZrw37rlefSzUP7z6SD2bB1ItZ3TrZeT8Ykfdybz7NwNdKxbgTLFguMj1psCIs0Z1awAIlIByPRpVMYEiW7RlWla9UG+n7iD7hvehdinrHAwPrE7+QRhocLVZYoGOpSzvLnF9A7wGVBRRF4BlgGv+jQqY4JItaMJdM9YALFPoQnjmDxtCr8kHQl0WKaQ2ZOSSpWyxQgNCZ5mam8Gyk0VkVW4ciYJ8FdV3ezzyIwJBu4jq6NiOVSxFX+ZM4hhm/7Hw3WP0rLtjYRYA7bJB7uDKM13lvPWIESkFpCoqu8BG4DOIlLG14EZExRypOoo1/BGwnpPonvF33n715IcnzqAQxu+cW1rDdjmIqlq0I2BABBXbrw8NnAlzIvBNRXol7gysjZQ1Vt9HdyFiImJ0YSEhECHYS4Tqsr0n/fy9RezeLvI25Rudz+yarw1YJuLduRkGukZmVxRIsKv5xWRVaoa42mdN43UmaqaLiJ3AO+q6qisHk3GXK5EhLtaVaNl1P0cX3aYMkvfILP9k5yp0habAdtcjNJFgyP/kjtvGqnTRKQvcDfwhbMs+H4TYwKgduoaqmyfBrFPcXrFhzz71mg27z96/h2NcbNqdwpvfP0rR08FxzwQWbwpIAbhGij3iqomikgUMMW3YRlTALg3YHd6hq0dRvHP1Nd5dfRYJvyQyPlu3xqTZfn2ZN5bsoOwkOBKj3feaFR1k6r+TVWnO68TVfX/fB+aMUEuRwN2k/ZdCek1ka7l/8eLn29i1ttPcHjj4uz72GRExt2ytyAxnt0pJ7iyVARFw0OD6j3iTS+mOiIyR0Q2icjOrIc/gjMmqHlI01G6/g30+NsbvNStAQtSKlH883v+nLHOejmZnCo3g9lxlPxtBdXLFQ+694g3vZiWAc8DI4G/4LrlFOI+M1wwsF5MJtgcOZFG6d9XwOw4tlfrTa09MxHr5WRySozn0KR+JFToTufUL/zeEy6vXkze3PAqqqqLcRUmu1X1BeC2/AzQmMKodLEwiIolpX5/av86mmkZN/Jr0ehAh2WCzOmqbZmpnel8YBLEDAmqLxDeFBCnRSQE2CYiD4tId6CEj+MypnBIjKfcpo/Z1WAot57+ilfeG8v4ZYlkZloDtnGJ2PsD9xf/jox2T0LCuD9vSQYBb8ZBPAIUA/4GvARcDwz0ZVDGFApuvZxqRMVypMENjJ4ziPvmZ1Jm9QG63347UtPSdFzWnPeI9JxIaFQs1IrNltol0LzpxbRSVY+rapKqDlLVO1X1R28OLiJdRGSLiGwXkac9rK8mIktEZI2IrBeRW93W/cPZb4uI3Hxhv5YxQSBHL6fS9W+gRP8pPNEwlbJ1r0PmDLIG7MvdvtUsix7B4z+XIj0j0/Ve6TnR9d4JAt40Ui8CeqrqYed1WWCGqub5oe2kCN8KdAaSgJVAX1Xd5LbNWGCNqr4vIvWB+apaw3k+HWgJXA18A9RV1YzczmeN1KbASYzn9PS7+aFsN64/+jnSa2JQfGs0/vXUnHV8++sBEv51Y0DOf6mN1OWzCgdwTRgEVPRiv5bAdlXdqapngBlAtxzbKFDKeV4a+M153g1XIXRaVROB7c7xjCk8omJZU/EOOv0+kSkZN7AhvEmgIzIBsOvgCaLKB1eSvizeFBCZzjzUAIhIdZzJg86jMrDX7XWSs8zdC0B/EUkC5gPDLmBfROQ+EUkQkYQDBw54EZIxQSQxnutS5rK30cP85cwCXnt/LGO+30GGNWBfVhKTU6lxRfFAh+GRNwXEM8AyEZkiIh8D8cA/8un8fYGJqloFuBWY4vSY8oqqjlXVGFWNqVChQj6FZIwfuDVgV73zFcJ6T+L9iFF8//UnrNyVEujojJ8cP53OgWOnqVG+gBYQqroAaAbMxHWbqLmqfu3FsfcBVd1eV3GWuRsCzHLOswKIBMp7ua8xBVeOBuwS13aiRL8pvNEmnetqXgHL3mLfmhz/ZkGUgsHkj0OpZ6hdsQS1KwbnyAGvvq2r6kFV/QKIUdWDXh57JVBHRKJEJBzog2suCXd7cM1Uh4hci6uAOOBs10dEIpzkgHWAn708rzHBz0OaDqnZgSq3/xOAPUWvoejcIYweP55jp9Ksl1MhVbVcMb55vAM3N7gq0KF4dKGpA7t6u6GqpgMPA18Dm4FZqrpRRIaLSNZx/g7cKyLrcPVailOXjbhqFpuABcDQvHowGVPYXB19E982ep3eu59jzogHSZsxMGj6xpvLx3m7uWbbWGSNqjb1YTwXzbq5msLot8+e5ep17/BOenfSO/yTxzvXDXRIJh/9e/5mkg6f5L27AlczvKRuriLS1u1lcw/LjDG+kBjP1dumcqbtEwyO+Jaqh+0LUGGzavchDhw7HegwcuXNLaZRWU9UNTPnMmOMD7j1cgrv/Cwl+n9Mj8R/QWI8S7cdYNW059Gd35+7jzViFyi7klOJCtIurpBHLiYRaQ20ASqIyONuq0oBob4OzJjLWo5eTkTFulKF71vN3N9K89uG4ozdPoCMOydQpsEN2We3MwXCsVNpHDx+Jmi7uELeyfrCcWVtLQKUdFt+FOjhy6CMuex5StgXFQtRsbyRqUyqXIqHFoTwzuw4dm4cQM1dM6wRu4DZnXwCIGhHUUMeBYSqfg98LyITVXU3gDOIrYSq2qzsxgRISIgwqG0UbWvfz1cTtnHXpvdIajSMKlY4FDjt65SndsWS598wQLxpg/i3iJQSkeLABmCTiDzp47iMMedR98Qa+oYsYmPt+6m8YxokxnP0VFqgwzJeali5NFOGtAraQXLgXQFR36kx/BX4CogCBvgyKGPMebjNI9Cg/+tIz4lkzBzI4/83iqUT/kXGDmvADnYXMsQgULwpIMJEJAxXATFPVdPwLlmfMcZXPDRin+o+jm4V/sfobaU4/nF/9q9d6Fpno7CDUu+xP/LojDWBDiNP3swoNwbYBawD4p1srtYGYUwgeWjELl6vE3+p1wnW/cbjn4XyxmeD+eWXu2i0f441YAeh7X8cp1aF4O3BBOcpIJxG6d9VtbLbsj24ph01xgShvzS5mpgaD/HdR9u5Y8cHEPuUFQ5B5sjJNFJSzwRtmu8sed5icgbGPZVjmTp5lowxQapSykq6Z3xNersnIGEciQlf8fXG/wU6LOPYdTAVIKjHQIB3t5i+EZEncKX7Ts1aqKqWtN6YYOTWgF0kKhZqdaDCx/3558mHWdzsZp77SwNKRHjzr298ZVey66M0KsgLCG8aqXsDQ3FNFLTKeVhSGGOClYcG7Ii+U3ig9hHmrEpiyojH2LLiy+z7WC8nv7q6TFF6xVShWrngHSQHF5jNNZhZNldjzi9hVwpTpn/Mc6deJ+nG0TRp3zV7mg5rq7js5JXN9bz1TKeL64NA1jvnO+ADp7urMaYAialRjmsef4iFX1Sg+4pHIe0XNGGcK8+TFQ5+c/D4acoVCyckRAIdSp68ucX0Pq4036OdR3NnmTGmACoRUYQ77uyLxAyB+NeZnHYDHyVVITOzcNxNKAhufPN7np+3MdBhnJc3BUQLVR2oqt86j0FAC18HZozxocR4SBjHydZ/546Mr1n81Rzu+uhHkg6dCHRkhd7hE2c4fCIt6NsfwLsCIkNEamW9EJGagE3/aUxB5dbmUPTm5yjRfwoTSoym6L7l3PLWUtbPfNHmmvChxALSxRW8KyCeBJaIyHci8j3wLa65pI0xBVGOXk5SswORfSczsl0m11YqxZfJlWDOIFehAJaqI5/92cU1+GsQeU0Y9CiwHPgeqAPUc1ZtUdXgnSPPGJO3XOaaKBMVy/QblNQzMcj+a8iYNZA9UX2Jsrkm8lXiwROECFQtALeY8urFVAV4C7gG+AX4AVeBkQRYAWFMIRQaIpSKDIOoWOJLdeX6Te/xTYWBtKrUmuCdtaBgia1TnlKRRYgoEvwTc+Z6i0lVn1DVNsBVwD+AFGAQsEFENnlzcBHpIiJbRGS7iDztYf1IEVnrPLaKyGG3dRlu6+Zd6C9mjLkEifF0PPY5yysPpukfn/LPN0fz487kQEdVKMTUKMc97WsGOgyveDPeviiueahLO4/fcNUo8iQiocB7QGdctY6VIjJPVc8WLqr6mNv2w4Cmboc4qarRXsRnjMlPbqk62kTFsmXFjbz09QM8+NFpSjc7xbXNO2a/3ZQY72rX8HTrymSjqqzZe5jaFUu4ampBLtcahIiMFZEfcOVgao3r9lJPVY1xurqeT0tgu6ruVNUzwAygWx7b9wWmex+6McYncjRi12t9GxF3Teb+2keo3qgdzI77c0Iia8C+ICmpZ7hj9HJmrdwb6FC8klcNohoQAWwD9uGqBRy+gGNXBtyvQhLQytOGzhwTUbh6SGWJFJEEIB14TVXnetjvPuA+gGrVql1AaMaYXHmoCRStez0d67qy/J+5YzwnP+7Pliq9aJH8mY3CvgAFJUlflrzaILrgGhA3wln0d1y3iRaKyIv5HEcfYI6quo+vqO7kB7kLeMt9LIZbjGOdGk1MhQoV8jkkY4wnp6u0ZXm5v9Jy70fM4iZ2lLDag7d2HXQNRCwIYyDg/PNBqKpuAObjmo/6B6AW8IgXx94HVHV7XcVZ5kkfctxeUtV9zs+duPI/NT13N2OMv5Xcv4JbTn7J1noP0vnEF7z4zhgm/JBoqTq8sCs51dXFtWzwd3GFvNsg/iYiM5wZ5L4Hbgd+Be4Aynlx7JVAHRGJEpFwXIXAOb2RROQaoCywwm1ZWRGJcJ6XB9oCXvWcMsb4kNso7Lp9X0N6TOS9sHfYsmI+6ZnqGm2dNcDOfR8bhQ24RlFXKVuM8CLejFEOvLzaIGoAs4HHVHX/hR5YVdNF5GHgayAUGK+qG0VkOJCgqlmFRR9ghmbPO34t8IGIZOIqxF5z7/1kjAmQHA3YZRveiBabwvO7EwgvEsLxKxoTNv1uwvtMQmp2yJ5K3PBAh1ocPF5whpHZfBDGmHzz3pLtLFv0KWMj3yWk5RCKr59ko7CDXF7zQRSMeo4xpkB4sEMtbrilB5PSb6D4j2+yrVpvKxwcx06l8cX63zhwrODUIKyAMMbkm5AQ4Z4qSTxY7DtmFO1Duc1T+HzujECHFRR+/d8xHp62hg2/HQl0KF6zmcuNMfnHaXMI7T2JHtXa8cW8mfxlyz8g8WpOV21bIPIP+UpWmu+oKwpGF1ewAsIYk5/cGrGLAH/t3hcSK5OZtJpP587laNnG3NWnHyWz0kxcRmk6dh1MpUiIUKVs0UCH4jW7xWSMyT/tHj23zSEqlow2f0OubkaPxH/xz/+MZvn2g5ddmo5dyalUKVuUIqEF52PXahDGGJ8LCw2hT+/+bP2pLC999QCTJq6hSeQSQntNJPIyacROPHiiwIygzmIFhDHGb+q2uo20o/fxyA8jmEBPelRrR2Sgg/KTsQOacyYjM9BhXJCCU9cxxhR8ifGErZkAsU8RF/YtJfev4HR6BmO+38GptMI91X3VcsWoVaFEoMO4IFZAGGP8w31UdadnkF4TYXYc65d+zmtf/crkNx5j+0/zz92nIKfpcFKP7DxwnA/jd5J8/HSB+p2sgDDG+EeONB1ExULPibQI28WUIS1ZpzUpO/8+Zsz6mNPpGYWjEbtyM5gdx+6EBbwyfzPpOwrW72SpNowxQeHoqTSmzfiYnonPklC+Ozef/LJwpOlIjOfE1AGMP92JoSW/D7r5MyzVhjEm6JWKDOOBuEEcbXA3NydPhpghpF7dhjPpBath9xxRsXwRfgsPh36KxAwJqsLhfKyAMMYEj8R4onbNgNinIGEck6ZN5q/v/cDm/UcDHdlFO7l1CTekfsHyyoMhYdy56dCDWKHu5pqWlkZSUhKnTp0KdCimEIqMjKRKlSqEhQX/5PMFgnsjdlQsRLXnnul3sy79Ebq+e4yJdZdzXbvOhNbqkH2fYB6JnRhPkU8H80jGI9zfMQ6KdM/+Owa5Ql1AJCUlUbJkSWrUqIGIBDocU4ioKsnJySQlJREVFRXocAoHD43Y4X0nMzJxJf/v90q8+0spGu7qz4lu46gUfVPBmGti32rCek9ibOU2hIWGQKirYZ59qwtEAVGoG6k3b97MNddcY4WD8QlV5ddff+Xaa68NdCiXhfm/7GfRl7P5j7xFSIshrts1BeSbeDC7rBuprXAwvmLvLf+6tVElRjw1zFU4xL9OfOmubCsW3FPVHz+dTq8PVrBs28FAh3JRCn0BYYwpPEJ3L4WEcRxu8SiN9s9h+KgPeP+7HaQHaQqLlbtS+DkxhYL6XcIKCB9KTk4mOjqa6OhorrrqKipXrkx0dDRlypShfv36fo1l7ty5bNr057Tezz33HN98880FH2fXrl00bNjQ4/Jp06adfZ2QkMDf/va3izoWwMSJE/ntt988ruvYsSP+HPOSV5zGj9zaHMrc9iIhvSYxOvwd4hd+wsdvPs6+NQvO3T7AI5Z/3JFMeGgIzaqVDWgcF8sKCB+64oorWLt2LWvXruWBBx7gscceO/s6JCT/L316enqu63IWEMOHD+fGG2/Mt3PnLCBiYmJ45513Lvp4eRUQ5jKVoxG7dP0bKNF/Cs82PcUPJ6pR/L/3krHje9e2QTIK+8edyURXLUPR8II5UVKh7sWUU+8PVpyz7PbGlRjQugYnz2QQN+Hnc9b3aF6FnjFVSUk9w4Mfr8q2bub9rS86loyMDO69916WL19O5cqV+e9//0vRokXZsWMHQ4cO5cCBAxQrVowPP/yQa665hl27djF48GAOHjxIhQoVmDBhAtWqVSMuLo7IyEjWrFlD27ZtGTp06Dn7p6SkMG/ePL7//ntefvllPvnkE1566SVuv/12evTowcqVK3nkkUdITU0lIiKCxYsXk5yczIABA0hNdc2C9e6779KmTZtcf5+nn36azZs3Ex0dzcCBA2natCkjRozgiy++4MCBA9x111389ttvtG7dmkWLFrFq1apcr8OXX35JQkIC/fr1o2jRoqxYsYKiRbNPsjJlyhTuuece0tPTGT9+PC1btiQlJYXBgwezc+dOihUrxtixY2ncuDEvvPACJUqU4IknngCgYcOGfPHFFwDccssttGvX7py/w6pVqxg8eDAAN91000X/nU0+8tCVVWp2oH7NDrx67DSHfr2WMp8MIr3ZICRhPKG9JwW0AfvoqTR+2XeEhzvVCVgMl8qnNQgR6SIiW0Rku4g87WH9SBFZ6zy2ishht3UDRWSb8xjoyzgDYdu2bQwdOpSNGzdSpkwZPvnkEwDuu+8+Ro0axapVqxgxYgQPPfQQAMOGDWPgwIGsX7+efv36Zbt9k5SUxPLly3nzzTc97t+mTRu6du3KG2+8wdq1a6lVq9bZfc+cOUPv3r15++23WbduHd988w1FixalYsWKLFq0iNWrVzNz5szz3i567bXXaN++PWvXruWxxx7Ltu7FF1+kU6dObNy4kR49erBnz548r0OPHj2IiYlh6tSprF279pzCAeDEiROsXbuW0aNHn/0gf/7552natCnr16/n1Vdf5e67777ov8OgQYMYNWoU69atO+8xTOBVKBlBVItbIGYIRZaN4P0THRmzp3JA2yaOnEijQ90KtK9TPmAxXCqf1SBEJBR4D+gMJAErRWSeqp69z6Gqj7ltPwxo6jwvBzwPxAAKrHL2PXQpMeX1jb9oeGie68sVD7+kGkNOUVFRREdHA9C8eXN27drF8ePHWb58OT179jy73enTpwFYsWIFn376KQADBgzgqaeeOrtNz549CQ0NzXP/3GzZsoVKlSrRokULAEqVKgVAamoqDz/8MGvXriU0NJStW7de9O+6bNkyPvvsMwC6dOlC2bJ/3o/1dB280bdvXwBiY2M5evQohw8fZtmyZWc/4Dt16kRycjJHj+Y9AtfT+Q8fPszhw4eJjXV9+xwwYABfffWVt7+uCZTEeEgYR+p1jzPw53Hc9/W1fLWhLSN6NKbOlSX9Hk7VcsWYMKil38+bn3x5i6klsF1VdwKIyAygG7Apl+374ioUAG4GFqlqirPvIqALMN2H8fpVRETE2eehoaGcPHmSzMxMypQpw9q1ay/oWMWLu2aputj9PRk5ciRXXnkl69atIzMzk8hI30zr4uk6eCNnF9O8upwWKVKEzMw/v0m6j6y/2PObIOPWgF08Khat25GJMwby8MEi3PbOUT5p/DONWl6f/ZaTj0dhHz2VRqnIgj3K3pe3mCoDe91eJznLziEi1YEo4NsL2VdE7hORBBFJOHDgQL4EHUilSpUiKiqK2bNnA66BWFm3ONq0acOMGTMAmDp1Ku3bt7+g/UuWLMmxY8fO2adevXrs37+flStXAnDs2DHS09M5cuQIlSpVIiQkhClTppCRkfdkLrkdH6Bt27bMmjULgIULF3Lo0PkrgnkdD2DmzJmAq3ZSunRpSpcuTfv27Zk6dSoA3333HeXLl6dUqVLUqFGD1atXA7B69WoSExPzPHeZMmUoU6YMy5YtAzh7TBPEcjRgS80ORPSdzFvtM7m54VWUqtUKZsehO/3TiH3kZBpNhy9i0vJdPjm+vwRLL6Y+wBxVvaAppVR1rKrGqGpMhQoVfBSaf02dOpVx48bRpEkTGjRowH//+18ARo0axYQJE2jcuDFTpkzh7bffvqD9+/TpwxtvvEHTpk3ZsWPH2e3Dw8OZOXMmw4YNo0mTJnTu3JlTp07x0EMPMWnSJJo0acKvv/56tpaSm8aNGxMaGkqTJk0YOXJktnXPP/88CxcupGHDhsyePZurrrqKkiXzrvLHxcXxwAMPEB0d7fFbfWRkJE2bNuWBBx5g3LhxALzwwgusWrWKxo0b8/TTTzNp0iQA7rzzTlJSUmjQoAHvvvsudevWzfPcABMmTGDo0KFER0dTWLINFGrtHj23QToqluKdnmBU36ZUj+kCPSeSOnUAP4//O+rjfEgrE1PIyFTqXeX/W1v5SlV98gBaA1+7vf4H8I9ctl0DtHF73Rf4wO31B0DfvM7XvHlzzWnTpk3nLDP+d+rUKU1LS1NV1eXLl2uTJk0CG1A+svdYwXEmPUO/fHuo6vOldNq/79ON+4747Fwvfb5R6zwzX0+eSffZOfILkKC5fK76sg1iJVBHRKKAfbhqCXfl3EhErgHKAu59UL8GXhWRrNbMm3AVMKYA2rNnD7169SIzM5Pw8HA+/PDDQIdkLkNhe5Zx66n5bL/2Ibps/phh79WmecduDL2+NuFF8vdmyoqdyTSrVobIsII5/iGLzwoIVU0XkYdxfdiHAuNVdaOIDMdVYs1zNu0DzHBKsqx9U0TkJVyFDMBwdRqsTcFTp04d1qxZE+gwzOXMrRG7dlQsxzZ3ZsycOP72QwinQ5Twmi3zrQH7yIk0Nu0/yqM3nP9WZrDz6UA5VZ0PzM+x7Lkcr1/IZd/xwHifBWeMuXzkaMQueW0n6P8xb+/8mZI1W6Kz41jc8HXa39SdiL0/XFIa8ZAQePmvDWlRo1y+hR8ol9VIamPMZcpTTSAqlpJOgbGxzds0XfQgs9cupHfIIsIuYRR2ycgw+rWqfgnBBo9g6cVkjDEB07DdXzjW8G76n5nJ+8c78Mqm8pw8c0GdKs9asOF/7D9SOMbTWAFhjDGJ8dRInMHpNn9nSMQSNvzwBfdOvvCMwYdPnOHBqauYnZDkgyD9zwqILMveOncy8XxIFxwaGkp0dDQNGzakZ8+enDhxItvyrMdrr70GuFJZ16tXjyZNmtC2bVu2bNkCcDZRXs6sqf4UDBlWA5F6+4UXXmDEiBF+PafxI7cG7IibnqN4/ylMLvk+/6zvGnybFj+Sk1uXnLuPh8+GH3emoAqta13h+7j9wAqILJWbud4kWYVEPo20LFq0KGvXrmXDhg2Eh4czZsyYbMuzHk8//Wcuw6lTp7Ju3ToGDhzIk08+CcDy5csB3xcQqpotLYW7YCggjMl3HubCDuszifqZ2wH4ZH8FTk27m/VLnY6XeXw2/LgzmciwEBpXKe2f2H3MCogsUc5k4rPj4NtX/uzFkI8jLdu3b8/27du93j42Nvbs9iVKlABcabWXLl1KdHT0OSOWhw4dyrx5rjdx9+7dz2Y5HT9+PM888wwAb775Jg0bNqRhw4a89dZbgKvQqVevHnfffTcNGzZk7969xMXF0bBhQxo1asTIkSOZM2fO2RTcuY1u/r//+z8aNWpEkyZNzhZ4H374IS1atKBJkybceeedZ2tQs2fPpmHDhjRp0uRsUryMjAyefPJJWrRoQePGjfnggw88Xpf09HT69evHtddeS48ePc4ec/HixTRt2pRGjRoxePDgs4kKa9SowcGDrikfExIS6NixI+CqGQwePJiOHTtSs2bNbPNXvPLKK9StW5d27dqdrcWZQiqXUdhZDdu1W93KS5FPUvmbh/jmvb+ROSsu18+GH3cmE1O9HBFFCvb4h7NyG0FX0B75NpJ68cuqz5dy/cwHxYsXV1XVtLQ07dq1q44ePVpVVUNCQrRJkyZnHzNmzFBV1Q4dOujKlStVVfX111/XXr16ZTvOkiVL9LbbbvN4runTp+sTTzyhqqotWrTQVq1aqapqXFycLliwQBMSErRhw4Z6/PhxPXbsmNavX19Xr16tiYmJKiK6YsUKVVVNSEjQG2+88exxDx06dE5sOc2fP19bt26tqampqqqanJysqqoHDx48u80zzzyj77zzjqqqNmzYUJOSkrId/4MPPtCXXnpJVV2jr5s3b647d+7Mdp7ExEQFdNmyZaqqOmjQIH3jjTf05MmTWqVKFd2yZYuqqg4YMEBHjhypqqrVq1fXAwcOqKrqypUrtUOHDqqq+vzzz2vr1q311KlTeuDAAS1XrpyeOXPm7HVKTU3VI0eOaK1atfSNN97w+HvbSOrLw8kz6bp07KOqz5fSj14crMu3Hzxnm8OpZ7T6//tCRy3eGoAILx55jKS2GoQ7J10wsU+5fuZsk7gIJ0+eJDo6mpiYGKpVq8aQIUOAc28x9e7d++w+Wd/Sf/jhhwu6992+fXuWLl3Kpk2bqF+/PldeeSX79+9nxYoVtGnThmXLltG9e3eKFy9OiRIluOOOO1i6dCkA1atX57rrrgOgZs2a7Ny5k2HDhrFgwYKzKcDz8s033zBo0CCKFSsGQLlyrj7gGzZsoH379jRq1IipU6eyceNGwJXALy4ujg8//PBsIsCFCxcyefJkoqOjadWqFcnJyWzbtu2cc1WtWpW2bdsC0L9/f5YtW8aWLVuIioo6m2dp4MCBxMef/+932223ERERQfny5alYsSK///47S5cupXv37hQrVoxSpUrRtWvX8x7HFG6RST/Q7tB/+b3p3+ihC6l8eOU525QuFsZP/7yBPi2rBSBC37BxEFncGqqIioWo9vlymymrILgQU6dOJSYm5oLPVblyZQ4fPsyCBQuIjY0lJSWFWbNmUaJEifMmx3NPxle2bFnWrVvH119/zZgxY5g1axbjx2cfs/jTTz9x//33A67pS3MTFxfH3LlzadKkCRMnTuS7774DYMyYMfz00098+eWXNG/enFWrVqGqjBo1iptvvjnPWC8k1TdkT/ftnuobzk33nde0reYy5fbZcGVULNroRkrPGQTlJvLihivoceoT6jfvgNTswJWlIv/cx4epxP3FahBZPDRU0XOia3kQOV8a7Ouuu4633nqL2NhY2rdvz4gRI86mBm/fvj1z587lxIkTpKam8tlnn3lMG37w4EEyMzO58847efnll8+mynY/d6tWrc7Wfrp27Urnzp2ZMGHC2faAlBRXZpRjx45RqVIl0tLSsqXN3rFjB61atWL48OFUqFCBvXv3cvPNN/P++++TlpYGwNatW89Oeepuz549rFjhSt01bdo02rVrR7169di1a9fZNpspU6bQoUMHwNUGkTXFadaEQnmJjY1l7ty5nDx5kmPHjvH555+fdx9TiHlIJU7PiaTtWcWaPYd5eU0kxz8ewEdTJrFw4/+CZj7s/GA1iCy5jLT0VTrgrFtPWbp06XK2q2te3NNqx8XFnTO9Z/v27Vm4cCG1a9emevXqpKSknC0EmjVrRlxcHC1buma5uueee2jatOk5s7jt27ePQYMGnf3W/e9//xv4MwW3p3miu3Tpwtq1a4mJiSE8PJxbb72VV199lZdeeolWrVpRoUIFWrVqdbaAefLJJ9m2bRuqyg033ECTJk1o3Lgxu3btolmzZqgqFSpUYO7cuedcg3r16vHee+8xePBg6tevz4MPPkhkZCQTJkygZ8+epKen06JFCx544AHAlW58yJAhPPvss2cbqPPSrFkzevfuTZMmTahYseLZ2fbMZSqXz4awqFg+aa9M+KESDy8M4c3tz7AzbT0c/MynqcT9SbSQ5LqPiYnRhITsA1s2b97MtddeG6CIzOXA3mMGYHdyKrvnPEPs/gmuNsxOzwQ6JK+JyCpV9XhP224xGWPMJap+dBWxR+blaweXYGAFhDHGXAr3Di6dnvlzPFUhKCQKfQFRWG6hmeBj7y0DFJgOLhejUDdSR0ZGkpyczBVXXHHerpDGXAhVJTk5mcjIyECHYgLNzx1c/KlQFxBVqlQhKSmJAwcOBDoUUwhFRkZSpUqVQIdhjM8U6gIiLCyMqKioQIdhjDEFUqFvgzDGGHNxrIAwxhjjkRUQxhhjPCo0I6lF5ACw+xIOUR44mE/h5CeL68JYXBfG4rowhTGu6qpawdOKQlNAXCoRSchtuHkgWVwXxuK6MBbXhbnc4rJbTMYYYzyyAsIYY4xHVkD8aWygA8iFxXVhLK4LY3FdmMsqLmuDMMYY45HVIIwxxnhkBYQxxhiPCn0BISJdRGSLiGwXkac9rI8QkZnO+p9EpIbbun84y7eIyM1+jutxEdkkIutFZLGIVHdblyEia53HPD/HFSciB9zOf4/buoEiss15DPRzXCPdYtoqIofd1vnyeo0XkT9EZEMu60VE3nHiXi8izdzW+fJ6nS+ufk48v4jIchFp4rZul7N8rYgkeNrfh3F1FJEjbn+v59zW5fke8HFcT7rFtMF5T5Vz1vnyelUVkSXOZ8FGEXnEwza+e4+paqF9AKHADqAmEA6sA+rn2OYhYIzzvA8w03le39k+AohyjhPqx7iuB4o5zx/Mist5fTyA1ysOeNfDvuWAnc7Pss7zsv6KK8f2w4Dxvr5ezrFjgWbAhlzW3wp8BQhwHfCTr6+Xl3G1yTofcEtWXM7rXUD5AF2vjsAXl/oeyO+4cmz7F+BbP12vSkAz53lJYKuH/0mfvccKew2iJbBdVXeq6hlgBtAtxzbdgEnO8znADSIizvIZqnpaVROB7c7x/BKXqi5R1RPOyx8Bf+SV9uZ65eZmYJGqpqjqIWAR0CVAcfUFpufTufOkqvFASh6bdAMmq8uPQBkRqYRvr9d541LV5c55wX/vL2+uV24u5b2Z33H58/21X1VXO8+PAZuByjk289l7rLAXEJWBvW6vkzj34p7dRlXTgSPAFV7u68u43A3B9Q0hS6SIJIjIjyLy13yK6ULiutOpys4RkaoXuK8v48K5FRcFfOu22FfXyxu5xe7L63Whcr6/FFgoIqtE5L4AxNNaRNaJyFci0sBZFhTXS0SK4fqQ/cRtsV+ul7hufzcFfsqxymfvsUI9H0RhICL9gRigg9vi6qq6T0RqAt+KyC+qusNPIX0OTFfV0yJyP67aVyc/ndsbfYA5qprhtiyQ1yuoicj1uAqIdm6L2znXqyKwSER+db5h+8NqXH+v4yJyKzAXqOOnc3vjL8APqupe2/D59RKRErgKpUdV9Wh+Hjsvhb0GsQ+o6va6irPM4zYiUgQoDSR7ua8v40JEbgSeAbqq6ums5aq6z/m5E/gO17cKv8SlqslusXwENPd2X1/G5aYPOar/Prxe3sgtdl9eL6+ISGNcf8Nuqpqctdztev0BfEb+3Vo9L1U9qqrHnefzgTARKU8QXC9HXu8vn1wvEQnDVThMVdVPPWziu/eYLxpWguWBq4a0E9cth6yGrQY5thlK9kbqWc7zBmRvpN5J/jVSexNXU1yNcnVyLC8LRDjPywPbyKfGOi/jquT2vDvwo/7ZIJboxFfWeV7OX3E5212Dq8FQ/HG93M5Rg9wbXW8jewPiz76+Xl7GVQ1Xu1qbHMuLAyXdni8Huvgxrquy/n64Pmj3ONfOq/eAr+Jy1pfG1U5R3F/Xy/ndJwNv5bGNz95j+XZxg/WBq4V/K64P22ecZcNxfSsHiARmO/8sPwM13fZ9xtlvC3CLn+P6BvgdWOs85jnL2wC/OP8gvwBD/BzXv4GNzvmXANe47TvYuY7bgUH+jMt5/QLwWo79fH29pgP7gTRc93iHAA8ADzjrBXjPifsXIMZP1+t8cX0EHHJ7fyU4y2s612qd83d+xs9xPez2/voRtwLM03vAX3E528Th6rjivp+vr1c7XG0c693+Vrf66z1mqTaMMcZ4VNjbIIwxxlwkKyCMMcZ4ZAWEMcYYj6yAMMYY45EVEMYYYzyyAsKYHETkGSdz5nonQ2crEXnUSbPgy/MOdwZHGhMUrJurMW5EpDXwJtBRXelEyuMamLUcV//ygwEN0Bg/shqEMdlVAg6qk07EKRB6AFcDS0RkCYCI3CQiK0RktYjMdnLlZM0N8LozP8DPIlI75wlEJFREJjrzCvwiIo85yyeKSA8RiXGbe+AXEVFnfS0RWeAkhVsqItf455KYy5UVEMZktxCoKq5Jh0aLSAdVfQf4DbheVa93ahX/Am5U1WZAAvC42zGOqGoj4F3gLQ/niAYqq2pDZ7sJ7itVNUFVo1U1GlgAjHBWjQWGqWpz4AlgdP78ysZ4ZtlcjXGjriyizYH2uCZtmulh9rLrcE0o9YNr6hDCgRVu66e7/Rzp4TQ7gZoiMgr4ElehdA4R6Y1rEpubnBpKG2C2c05w5QkzxmesgDAmB3WlCv8O+E5EfgEG5thEcE3E0je3Q7g/F5FQYJXzep6qPieuKT5vxpVTpxeunDl/nkCkIa7cUrGqmiEiIcBhp1ZhjF/YLSZj3IhIPRFxn38gGtgNHMM15SO4ksi1zWpfEJHiIlLXbZ/ebj9XqGpG1i0jp3AoD4So6ie4blU1c9sXESmDq/Zxt6oeAFcabCBRRHo624i4zSNtjC9YDcKY7EoAo5wP6XRcWTDvwzXN5AIR+c1ph4gDpotI1m2ef+HKNApQVkTWA6ed/XKqDExwagUA/8ixvhtQHfgw63aSU3PoB7wvIv8CwnBNu7nukn5bY/Jg3VyNyUcisgvrDmsKCbvFZIwxxiOrQRhjjPHIahDGGGM8sgLCGGOMR1ZAGGOM8cgKCGOMMR5ZAWGMMcaj/w8ErK5QMRT6EgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot theoretical and PEPit (numerical) worst-case performance bounds as functions of the step-size\n", "\n", "plt.plot(gammas, theoretical_taus, '--', label='Theoretical tight bound')\n", "plt.plot(gammas, pepit_taus, 'x', label='PEPit worst-case bound')\n", "\n", "plt.legend()\n", "plt.xlabel('Step-size')\n", "plt.ylabel('Worst-case guarantee')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
↩ Back to TOC
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.2 Example 2 : An accelerated gradient method for strongly convex objectives" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We consider the following convex minimization problem\n", "\\begin{equation}\n", "f_\\star \\triangleq \\min_x f(x),\n", "\\end{equation}\n", "where $f$ is $L$-smooth and $\\mu$-strongly convex.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The package PEPit allows to compute a worst-case guarantee for an **accelerated gradient** method (a.k.a., **fast gradient**, or **Nesterov's accelerated** method). That is, it computes the smallest possible $\\tau(n, L, \\mu)$ such that the following guarantee holds (for all initialization and $L$-smooth $\\mu$-strongly convex function)\n", "\\begin{equation}\n", "f(x_n) - f_\\star \\leqslant \\tau(n, L, \\mu) \\left(f(x_0) - f_\\star + \\frac{\\mu}{2}\\| x_0 - x_\\star\\|^2_2\\right),\n", "\\end{equation}\n", "where $x_n$ is the output of the accelerated gradient method under consideration, and where $x_\\star$ is the minimizer of $f$.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The accelerated gradient method under consideration is described as follows, for $t \\in \\{0,1, \\ldots, n-1\\}$\n", "\\begin{equation}\n", "\\begin{aligned}\n", " x_{t+1} & = y_t - \\alpha \\nabla f(y_t),\\\\\n", " y_{t+1} & = x_{t+1} + \\beta (x_{t+1} - x_{t}),\n", " \\end{aligned}\n", "\\end{equation}\n", "with $\\kappa = \\frac{\\mu}{L}$, $\\alpha = \\frac{1}{L}$ and $\\beta = \\frac{1 - \\sqrt{\\kappa}}{1 + \\sqrt{\\kappa}}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Theoretical guarantee" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A known worst-case guarantee is provided by \n", "\\begin{equation}\n", "f(x_n) - f_\\star \\leqslant \\left(1-\\sqrt{\\kappa}\\right)^n \\left(f(x_0) - f(x_\\star) + \\frac{\\mu}{2}\\|x_0 - x_\\star\\|^2_2\\right).\n", "\\end{equation}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot worst-case guarantees as functions of the iteration count" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Set the parameters\n", "L = 1 # smoothness parameter\n", "mu = 0.1 # strong convexity parameter\n", "gamma = 1 / L # step-size\n", "\n", "# Set a list of iteration counter to test\n", "n_list = np.array([1, 2, 4, 6, 8, 10, 15])\n", "\n", "\n", "# Compute numerical and theoretical (analytical) worst-case guarantees for each iteration count\n", "pepit_taus = list()\n", "theoretical_taus = list()\n", "for n in n_list:\n", " pepit_tau, theoretical_tau = wc_accelerated_gradient_strongly_convex(mu=mu,\n", " L=L,\n", " n=n,\n", " verbose=verbose,\n", " )\n", " pepit_taus.append(pepit_tau)\n", " theoretical_taus.append(theoretical_tau)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEMCAYAAADal/HVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABKCUlEQVR4nO3dd1yVdfvA8c/FEJy4Ry5wkYoCigvcmqNhablKc1RqpWXzaTy/1vOUNp7SsjTLVbkbllZqbsU9cOVWVNTcAwf7+/vjPhCawAEOnANc79frvOTcnPu+LxC4znddXzHGoJRSSqXFzdkBKKWUcm2aKJRSSqVLE4VSSql0aaJQSimVLk0USiml0qWJQimlVLo0USillEqXJgqllFLp8nB2ABkRkaLAF0AcsMIYM93JISmlVIHilBaFiEwWkTMisuuW411EZJ+IHBSRV2yHewDfG2OeALrlerBKKVXAOavraSrQJfUBEXEHPge6AvWAviJSD6gCHLe9LDEXY1RKKYWTup6MMatExPeWw02Bg8aYwwAiMgu4H4jCShYR2JnYypYta3x9b728Ukqp9GzZsuWcMabcrcddaYyiMn+3HMBKEM2AT4FxInIPMD+tk0VkCDAEoFq1amzevDkHQ1VKqfxHRI7e7rgrJYrbMsZcAwbZ8bqJwESAkJAQLYmrlFIO4krTY08AVVM9r2I7ZjcRuU9EJl6+fNmhgSmlVEHmSoliE1BbRPxEpBDQB/glMxcwxsw3xgzx8fHJkQCVUqogckrXk4jMBNoCZUUkCnjTGDNJRIYDiwB3YLIxZrcz4lMqt8THxxMVFUVMTIyzQ1EFiLe3N1WqVMHT09Ou1ztr1lPfNI7/BvyW1euKyH3AfbVq1crqJZTKVVFRURQvXhxfX19ExNnhqALAGMP58+eJiorCz8/PrnNcqesp27La9TRh5SHWHjp307G1h84xYeUhR4an1D/ExMRQpkwZTRIq14gIZcqUyVQrNl8liqxqWMWH4TO2pSSLtYfOMXzGNhpW0bEOlfM0SajcltmfOZefHpsbQmuWZdzDwQyfsY22/uVYvvcMnz/SiNCaZZ0dmlJKOV2+alFkZ3psaM2ydA+6gx+3niAmPpENhy9wJloHGFX+dv78eYKCgggKCqJixYpUrlyZoKAgSpYsSb169XI1lnnz5vHnn3+mPH/jjTdYsmRJpq8TGRlJQECAI0NzmGLFiuXq/aZOncrw4cOzfZ18lSiyMz127aFz/BRxku7BlUlIMoxdeoCWo5fz/JwIjl+4ngPRKpU5OTGWVqZMGSIiIoiIiGDYsGE899xzKc/d3Bz/5yEhISHNz92aKN555x06duzo8BhyS3pfa16TrxJFViWPSYx7OJhPegcxbXBTfAp70ta/HH/8eZrk7rxzV2NJTNJF38o5cnssLTExkSeeeIL69evTqVMnbty4AcChQ4fo0qULjRs3plWrVuzduxew3sm3b9+ehg0b0qFDB44dOwbAwIEDGTZsGM2aNePll1++7flr167ll19+4aWXXiIoKIhDhw4xcOBAvv/+ewA2bdpEaGgogYGBNG3alOjoaCIjI2nVqhWNGjWiUaNGrF27Nt2vZ8WKFdx7770pz4cPH87UqVMB8PX15eWXX6ZBgwY0bdqUgwcP3hR7SEgIderUYcGCBSnfm5deeokmTZrQsGFDvvzyy5R7tGrVim7duqXZInvuueeoX78+HTp04OzZswBERETQvHlzGjZsSPfu3bl48SIAbdu2TSlHdO7cOZJr2E2dOpUePXrQpUsXateuzcsvv5xy/SlTplCnTh2aNm1KeHh4Rv/NdtExCsAtfCzftG9KgG1MIrRmWaZ3jOXqoZ/59PW38PZ0B+CFOds5dPYqA0N96dWkKiW87ZuDrJS9en+57h/H7m1Yif4tfAmuWoryxb14dNJGKpTw4vSVWGqVL8aJi9Yf8AvX4njyuy03nTt7aIssx3LgwAFmzpzJV199Ra9evfjhhx/o168fQ4YMYcKECdSuXZsNGzbw1FNPsWzZMkaMGMGAAQMYMGAAkydP5plnnmHevHmANQ147dq1uLu706FDh9ue361bN+69914eeuihm+KIi4ujd+/ezJ49myZNmnDlyhUKFy5M+fLl+eOPP/D29ubAgQP07ds3WzXefHx82LlzJ9988w0jR45MSQqRkZFs3LiRQ4cO0a5dOw4ePMg333yDj48PmzZtIjY2lrCwMDp16gTA1q1b2bVr122nnl67do2QkBA++eQT3nnnHd5++23GjRvHo48+ymeffUabNm144403ePvttxkzZky68UZERLBt2za8vLzw9/dnxIgReHh48Oabb7JlyxZ8fHxo164dwcHBWf6eJMtXiSKr6yiat7oL5g6EMu5QpzMcDScg/FnoORVsSQKgb9OqTF4TyX9/3cMnf+ynZ0hVBob64lu2qEO/DqXS4lPYkwolvDhxKYbKJb3xKZxzb1b8/PwICgoCoHHjxkRGRnL16lXWrl1Lz549U14XGxsLwLp16/jxxx8B6N+//03vcnv27Im7u3u656dl3759VKpUiSZNmgBQokQJwPqjO3z4cCIiInB3d2f//v3Z+nr79u2b8u9zzz2XcrxXr164ublRu3ZtatSowd69e1m8eDE7duxIafFcvnyZAwcOUKhQIZo2bZrm+gQ3Nzd69+4NQL9+/ejRoweXL1/m0qVLtGnTBoABAwbc9P1JS4cOHUjuZq9Xrx5Hjx7l3LlztG3blnLlrAKwvXv3zvb3BfJZojDGzAfmh4SEPJGpE/1aW0nhuwfBqwQkxkLv6dbxVLoEVKJLQCV2Rl1mSvgRpm84SvkSXjzVthZJSQYRneqosie9FkDhQu4827E2w2ds45n2tfhuwzGe7Vg7ZXZe6aKFstWCuJWXl1fKx+7u7ty4cYOkpCRKlixJREREpq5VtKj1Ziqr59/OJ598QoUKFdi+fTtJSUl4e3un+3oPDw+SkpJSnt+6jiD1725aHyc/N8bw2Wef0blz55s+t2LFipSv1R4Z/b1IHfOt8d76/5OTYyI6RpHMr7XVmrh+DmKjYenbcHjFbV/aoIoPH/cOIvxf7enXvDoA83ecpOvY1czZdJyYeN1fSTle6rG05zv5p0zpvnWAOyeVKFECPz8/5s6dC1irfLdv3w5AaGgos2bNAmD69Om0atUqU+cXL16c6Ojof5zj7+/PqVOn2LRpEwDR0dEkJCRw+fJlKlWqhJubG99++y2Jien/3lWvXp0///yT2NhYLl26xNKlS2/6/OzZs1P+bdHi74Q7d+5ckpKSOHToEIcPH8bf35/OnTszfvx44uPjAdi/fz/Xrl3L4LtnJcrkVsiMGTNo2bIlPj4+lCpVitWrVwPw7bffprQufH192bLF6k5MPi89zZo1Y+XKlZw/f574+PiU73N2aaJIdmQVHF0LrV6EQsXg4lH45n6Ydh8c33TbU8qX8E4ZpyjubTXOXv5hB6Gjl/G/xfv4aNE+XfGtHGZH1GXGPRyc0oJIXv+zIyp3qyVPnz6dSZMmERgYSP369fn5558B+Oyzz5gyZQoNGzbk22+/ZezYsZk6v0+fPnz44YcEBwdz6NDfvyOFChVi9uzZjBgxgsDAQO666y5iYmJ46qmnmDZtGoGBgezduzfDd/JVq1alV69eBAQE0KtXr3/03V+8eJGGDRsyduxYPvnkk5Tj1apVo2nTpnTt2pUJEybg7e3N448/Tr169WjUqBEBAQEMHTrUrnf0RYsWZePGjQQEBLBs2TLeeOMNAKZNm8ZLL71Ew4YNiYiISDn+4osvMn78eIKDgzl3LuM3BJUqVeKtt96iRYsWhIWFUbdu3QzPsYcYk39m8aQao3jiwIED9p94ZJU1RtFzqtWySH5evwfs/slqZfjfDe1eh4ppz882xrD+8AUmhx9hyZ7TVCtVhOjYhJRf7tTvCHUxnwLYs2ePw36ZVdb5+vqyefNmypa9+fdy4MCBtx1gzw9u97MnIluMMSG3vlbHKABObP07ScDfYxYntsKz22HDeAj/DCa0hIAHod1rUKbmPy4jIrSoWYYWNctw9Pw1zl2NJTYhiaenb6W4tycXrsUxvp+u+FZK5S35qkWRLCQkxDh8K9TrF2DtZ7BhAiTEQnA/aPMy+FTJ8NRXf9zBzI3WLq+VSxZmQGh1eodUw6eITq8t6LRFoZwlMy0KHaOwV5HS0PFNeCYCmjwOETPg00aw8DW4lnbf4dpD51i0+zTD29WimJcHJQp78N5ve2k+ail/XdYSIUop16eJIrOKV4C7P4BntkLDnla31NhAWPYuxNw8qJh6TOLFzv5MfLQxp6/E8l73AJ5uV5OKPtZ0vinhR1h94Cz5sXWnlMr7NFFkVclqcP/n8NQGqH0XrPoAxjSENWMgzqoNldYslSsxCQxvXxuAuIQkvl59hP6TNtLpk1XM2HCMG3E6vVYp5TryVaLITvXYLCtXxxr4HroKqjaFJW/Cp0Gw8SuGyTxC3f686eWhbn8yzH1+yvNCHm4se7EN/+sZSCEPN177aSctRi9l6Z7Tufc1KKVUOvJVoshO9dhsqxQIj8yFwYugTC347UVr4HtmXzi0wnpN8rTbyo1uOtXLw50HG1dhwYiWzBnaguZ+ZahV3ipHvOfUFbYdu5i7X4sqUNzd3QkKCiIgIICePXty/fr1m44nP0aPHg1Yher8/f0JDAwkLCyMffv2AdaCO7BqI82YMcMpX8vUqVM5efKkU+6dzBllzt966y0++uijHLt+vkoULqFacxj4K/T7EUrcAXFX4bseVoJIvVbjNkSEpn6lmdC/MdXLWIuHPl9+kO5frKX7F+HM336S+MSk256rCoA1Y6w3G6kdWWUdz4bChQsTERHBrl27KFSoEBMmTLjpePLjlVdeSTln+vTpbN++nQEDBvDSSy8BpFRvzelEYYy5qRRHaq6QKPIjTRQ5QQRqdYAhK6DXt+DtYy3c8yxifZwJox9syNvd6nPxWhwjZm6j9QfLmb7haM7ErVxb5UbWm43kZJFGCzU7WrVqlVJi2x6tW7dOeX3ypjyvvPIKq1evJigo6KYVzgBPP/00v/zyCwDdu3dn8ODBAEyePJnXX38dgI8//piAgAACAgJSKqhGRkbi7+/Po48+SkBAAMePH2fgwIEEBATQoEEDPvnkE77//ns2b97MI488QlBQUEpZ9NTef/99GjRoQGBgYEri++qrr2jSpAmBgYE8+OCDKS2quXPnEhAQQGBgIK1bW2/u0iovfquEhAQeeeQR6taty0MPPZRyzaVLlxIcHEyDBg0YPHhwSkFEX1/flJXXmzdvpm3btoDVUhg8eDBt27alRo0afPrppyn3ePfdd6lTpw4tW7ZMadXlFLsW3IlIKeAO4AYQaYzRt7X2EIHCJW2J4y44uAS+bA1Bj0D7/4MSlTK8RDEvDwaE+tK/eXWW7zvD5PAjnIuOAyAhMYnI89dTuqlUHvf7K/DXzvRfU7wSfNvd+jf6FJS7E1a8bz1up2ID6DrartsnJCTw+++/06VLFwBu3LiRUj0W4NVXX02pfJps/vz5NGjQ4KZjo0eP5qOPPkop051aq1atWL16Nd26dePEiROcOnUKgNWrV9OnTx+2bNnClClT2LBhA8YYmjVrRps2bShVqhQHDhxg2rRpNG/enC1btnDixAl27doFwKVLlyhZsiTjxo3jo48+IiTkH0sB+P333/n555/ZsGEDRYoU4cKFCwD06NGDJ56w1uj++9//ZtKkSYwYMYJ33nmHRYsWUblyZS5dugTApEmTblte/NZqsfv27WPSpEmEhYUxePBgvvjiC4YPH87AgQNZunQpderU4dFHH2X8+PGMHDky3f+XvXv3snz5cqKjo/H39+fJJ59kx44dzJo1i4iICBISEmjUqBGNGzdO9zrZkWaLQkR8ROQ1EdkJrAe+BOYAR0Vkroi0y7Go8ovUpUH6fQ99Z4KHN+yYDZ81sn654+zbPc/NTehQtwLTH2/OiPZWGfXFf56m48creXTyRpbvO0OSbqqU/3mXtJLE5ePWv94ls33J5IQQEhJCtWrVeOyxx4B/dj2lThLJ79rDw8Mz1TeenCj+/PNP6tWrR4UKFTh16hTr1q0jNDSUNWvW0L17d4oWLUqxYsXo0aNHSrG86tWr07x5cwBq1KjB4cOHGTFiBAsXLkwpPZ6eJUuWMGjQIIoUKQJA6dKlAdi1axetWrWiQYMGTJ8+nd27dwMQFhbGwIED+eqrr1IKDi5evJhvvvmGoKAgmjVrxvnz57lduaCqVasSFhYGWOXE16xZw759+/Dz86NOnTqAVU581apV/zj3Vvfccw9eXl6ULVuW8uXLc/r0aVavXk337t0pUqQIJUqUoFu3bhleJzvSa1F8D3wDtDLGXEr9CRFpDPQXkRrGmEk5GF/edmtpEP+u1oD3wSVwMRJWvAdbploL+Rr0Aju3nnRzs0oTN/MrzYud6vDNuqMMmrKJGuWKMijMjz5NquLprr2KeY497/yT33y0fhk2T4K2/0pzzMteyQkhM6ZPn37bd+0ZSX53vnDhQlq3bs2FCxeYM2cOxYoVo3jx4umem7roX6lSpdi+fTuLFi1iwoQJzJkzh8mTJ9/0+g0bNjB06FDA2lY1LQMHDmTevHkEBgYydepUVqxYAcCECRPYsGEDv/76K40bN2bLli1plhe/1e1Kk6fHVcqJpyXNvybGmLuMMd/emiRsn9tijBnpaknCKdNj09Ny5D9/if1aw13vQK9vYNBCKF4RfhoKX7WDyMxtW1immBfD29dmzb/aM7ZPEMW8PJi85gjuth/K63H5Z89exc0t1PavW/+mHrNwEWmVC0/WvHlzxowZQ+vWrWnVqhUfffRRSknyVq1aMW/ePK5fv861a9f46aefbluu/Ny5cyQlJfHggw/y3//+l61bt/7j3s2aNUtpDXXr1o277rqLKVOmpIwXJHc9RUdHU6lSJeLj45k+fXrKPQ4dOkSzZs145513KFeuHMePH7e7vPixY8dYt87arTC5nLi/vz+RkZEpYzpplRP/4YcfMvwet27dmnnz5nHjxg2io6OZP39+hudkR4ZvO8XST0TesD2vJiJNczSqLHLq9NisqN4CHl8KPb6Ca2dh6t0wux9cOJypyxTycOP+oMr8/HQY3w9rgZubcCMukdYfLOfp6VvZcvSCrvrOD9IrXpkDkrukkh+pZz2lp2HDhri7uxMYGPiPwWywkkFCQgK1atWiUaNGXLhwISUZNGrUiIEDB9K0aVOaNWvG448/ftutPE+cOEHbtm0JCgqiX79+jBo1Cvh7j+vbDWZ36dKFbt26ERISQlBQUEqX2X/+8x+aNWtGWFgYd955Z8rrX3rpJRo0aEBAQEDKft32lhf39/fn888/p27duly8eJEnn3wSb29vpkyZQs+ePWnQoAFubm4MGzYMgDfffJNnn32WkJAQ3N3d/3G9WzVq1IjevXsTGBhI165dU3b/yykZFgUUkfFAEtDeGFPXNrC92BiTs5FlQ44UBcxpcddh3eew5hNIjINmQ6H1S9ZgeBZcvhHPFysOMnPDMa7EJBBYxYfBLf3oGlCJQh7aLeUqtCigchZHFwVsZox5GogBMMZcBAo5IlCVSqEi0OYlq4ZUYB8raXwaDBu/gsT4TF/Op7Anr3aty/rXOvCfBwKIjk3g2VkR7DrpIt1ySqk8w55EES8i7oABEJFyWC0MlROKV4T7x1klQSrUt1Z4jw+F/YsgC91HRQp50L95dZY814bZQ5oTXLUkAG/P380rP+xg319p9yUrpRTYlyg+BX4CyovIu8Aa4L0cjUpBpYYwYD70nQUmCWb0gm8fgNO7s3Q5NzehWY0yKbMvBGFexAk6j1nFI1+v58W5EYQf0G1bnUHHj1Ruy+zPnF0bF4nInUAHQIClxpg9WYoul+TJMYr0JMTB5smwYhTEXoHg/ta2rMUrZOuyF6/FMXPTMb5Ze5S/rsTg7enG5IFNdNvWXHTkyBGKFy9OmTJlMpxCqZQjGGM4f/480dHR/1gomNYYRZqJQkRKZ3CzC9kJNiflu0SR7PoFWPURbPzSWrjX6nlo/hR4Fs7WZeMTk1i46y8SEpP4z6976FC3PPMjTvL+gw25P7iyg4JXtxMfH09UVNQ/5s4rlZO8vb2pUqUKnp4377KZlURxBGtcIvXbnOTnxhhTw2FRO1i+TRTJzh+CP96AvQvApyp0fMvay9sB70g/XryPT5cdRLAu16leRQa39KOJbyl9x6tUPpfpWU/GGD9jTA3bv363PHfZJFEglKkJfabDgAVQuBT88BhMuguOb8zWZdceOsd3G47xTPta+BT25N6Gd7D+yHl6fbmORydn79pKqbwrM0UBawPeyceMMa61HBRrZTZwX61atZwdSu7wa2VVqN0+C5a+YyWL+j2sFkap6pm61K1jEs1rlmH4jG2M6RXEycsxJNrKCyQlGSaHH+GB4MqULeaVwVWVUvmBPQvuHgeeBaoAEUBzYJ0xpn2OR5dF+b7r6XZir8LaTyH8U2uWVIunoOXz4J1xsTSACSsP0bCKz00D12sPnWNH1GWGtamZcmzL0Ys8OH4thdzd6BZ0B4PCfKl/Rx5ZCa+USlemxyhSnbgTaAKsN8YE2WZAvWeM6ZEzoWZfgUwUyS6fsFoXO2ZBkbJWTaDgR8HdrsajXQ6eucq0tZF8vyWKG/GJNK9Rmk/7BlO+uHfGJyulXFZ2VmbHGGNibBfxMsbsBfwdHaByEJ/K0ONLeGI5lK0DC56DL1vBwaUOu0Wt8sX4zwMBrH+1A692vRNjoExRqxtq67GLRMdkfiW5Usp12dOi+AkYBIwE2gMXAU9jzN05Hl0WFegWRWrGwJ758Mf/WWXNa90Fnf4L5e/M8NSsiEtIInT0UmLik+gZUoWBob4pW7oqpVxflruebrlIG8AHWGiMiXNgfA6lieIWCbGwcSKs/NDaw7vxQGj3GhR1/EK67ccvMSX8CAt2nCLRGDrWrcBzHetQ7w77xkqUUs6T5a4nEWkuIsUBjDErgRXAP+v+Ktfl4QWhI+CZbdDkMWuzpE+DIXyslUQcKLBqScb0CSb8lfY83bYWW45e5IqtK+ry9Xhi4hMdej+lVM6zp+tpG9DI2F4oIm7AZmOM43Z0dzBtUWTg7D5Y/H9wYBGUrG5tpFTvfocs2LtVTHwiXh5uiAhv/ryLBTtO8Ujz6vRrXk0Hv5VyMdkZzBaTKpsYY5Kwc/2FclHl/OGROdD/JyhUDOYOgDENYePXN7/uyCpYMyZbt/L2dE9Z0d21QSWCq5Xks2UHCBu9jOfnRLDrhJY9V8rV2ZMoDovIMyLiaXs8C2RuCzblmmq2h2Gr4b6xEHsZfnsBpt0Pl6P+3nazsuMajs1rlOHrAU1Y9kJbHmlWnUW7/uLbdUdTPp+YpFVUlXJF9nQ9lccqNd4eq9bTUmCkMeZMzoeXNdr1lAWx0dZU2p1zQTzAoxD0ng61cm5d5ZUYa8yifHFvth27yPAZ2xgY6kuvJlXxKeyZ8QWUUg6V5a4nY8wZY0wfY0x5Y0wFY8zDrpwkVBZ5FYcHv7a2YDUJEH8dFr0KkWty7JYlvD1vGqeoXKow7/62hxajlvLmz7s4cu6fm9YrpXKfPbOePhCRErZup6UiclZE+uVGcCqXHVkFO7+H1i9bieP6BZh6D/w0DK6ezdFbB1crxZyhLVgwoiVdAioyY+Mxun8RTmyCzpJSytnsGaPoZIy5AtwLRAK1gJdyMijlBMljEj2nWmU/+syApAQIeMhKHuNCYPMUSMrZXXADKvvwca8gwl9pz2d9g/HycCcpyTD0283M2nhMp9cq5QT2JIrkGU73AHONMbk6TUVEaojIJBH5PjfvW+Cc2GolCb/W1nO/1tBrGlRsAE+GQ4UAWDASJneCUztyPJzyxb1pVbscAOeuxnLswg1e+XEnLUYt5aNF+/ho0T7WHtKtW5XKDfYMZo8GHgBuAE2BksACY0yzDC8uMhmrJXLGGBOQ6ngXYCzgDnxtjBltx7W+N8Y8lNHrQAezc4QxsGM2LHodblyAZsOs1d1exXPp9ob1hy8wOfwIS/acRoBiXh5M6N9Yt25VykGyVcLDti3qZWNMoogUBYobY/6y47zWwFXgm+REISLuwH7gLiAK2AT0xUoao265xODkgXNNFC7ixkVY8ra1urt4RegyOscW66Xl6PlrzN0cRbMapXl2VgQh1Uux9tB5xvdrlNIKUUplnkNqPWXxxr5YLZDkRNECeMsY09n2/FUAY8ytSeLW66SbKERkCDAEoFq1ao2PHj2a1kuVIxzfBL8+B3/thFod4e4PoXTub3yYvHUrwB0+3jwa6kvfJtXwKaLTa5XKrOyszHa0ysDxVM+jbMduS0TKiMgEIDg5qdyOMWaiMSbEGBNSrpy+q8xxVZvAEyug8yg4th6+aAErP3B47aj0JG/dOrxdLYp5eVCySCFG/76X5qOW8t16faOglKM4I1FkijHmvDFmmDGmZkatDpXL3D2snfSGb4I6XWD5uzA+FA6vyPFbpx6TeLGzPxMfbcxfV2IY3aMB9wVWokZZq7z5yUs3WLn/LDndclYqP8uwZpOI3K6Gw2XgqDEmIQv3PAFUTfW8iu1YthW4PbNdRYk7rBlSB5bAby/CN/dDg57Q6V0oXiFHbrkj6vJNA9ehNcsy7uFgdkRd5oOHAlNeN2PDMcYtP0it8sUYFOZLj+AqFC7kniMxKZVf2TPraT3QCNgBCBAA7Mbal+JJY8ziDM735eYxCg+swewOWAliE/CwMWZ3tr6SVHQw24nib8CaT6yHR2Ho8H8QMhjcnPPHOTYhkV93nGJy+BF2nbhCySKePNrCl+fvquOUeJRyZdkZozgJBNv6/xtj7UVxGGvW0gcZ3HQmsA7wF5EoEXnM1goZDiwC9gBzHJkklJN5FramzT65Du4IsloYX3eAk9ucEo6Xhzs9GlVh/vCWzB3WghY1ynDy0o2Uz+/7K9opcSmVl9jTotiVeg1E6mMiEmGMCcrJADMjVdfTEwcOHHB2OMoY2PUDLHoNrp2FJo9D+3+Dt49Tw0pKMri5CbtPXuaeT9cQVLUkg1v60TWgIp7uLj9sp1SOyfL0WBGZDVwAZtkO9QbKAv2BNcaYJg6ONdu068nFxFyGZf+FjV9BsfLQ+T0IeDBX117czrXYBH7YGsWU8EiOnLtGxRLe9G9RnQGhvhTz0i1XVMGTnURRGHgKaGk7FA58AcQARYwxVx0ca7ZponBRJ7ZapcxPRUCNtnD3/6Cs8yceJCUZVuw/w+Q1kWw/fom1r7anuLcn1+MSKFJIE4YqOLK7MrsQ4I+1H8U+Y0y840PMPu16ygOSEmHzZFj6DiTEQMvnoOXz4Oka26KevxpLmWJeGGPoOnY15Yp7MTjMjzZ1yuHm5twWkFI5LTstirbANKzKsYI1tXWAMWaVw6N0EG1R5AHRp2Hx69ZGSaX84J6PrBXeLiIuIYmvVh/mm3WRnL4SS42yRa3ptY2qUFS7pVQ+lZ1EsQVr+uo+2/M6wEzbDCiXpIkiDzm8An59Ac4fhHoPQJdR1roMFxGXkMTvu04xec0Rtkdd5pPegXQPruLssJTKEdlJFDuMMQ0zOuZKNFHkMQmxED4WVn0E7oWs/TCaPGGt/HYRxhi2HrtEQOUSeHm48/Xqw2w9dpHBYX40rl4KcfLAvFKOkJ11FJtF5GsRaWt7fAW45F9hEblPRCZevpyrW2ao7PLwgjYvw9ProVozWPgKfNUWolznx0xEaFy9FF4e1sJBY2DNgXM8NGEd938ezk/boohLyNlNnZRyFntaFF7A0/w962k18IUxJveqv2WStijyMGPgz5+tZBH9FzQeCB3fhMKlnB3ZP1yPS+DHrSeYEn6EQ2evcW/DSox7+HYVb5TKG5xWZtwZNFHkA7HRsHwUbBgPhUtDp/9CYB+nr724naQkw6oDZ/Ep7ElwtVKcunyDMX8cYGCYL3UrlXB2eErZLdOJQkR2Yk2HvS0do1C54tQO+PV5iNoEvq3gnv9BOX9nR5WuRbv/4tlZ24iJTyK0Zhkq+XjTI7gKYbX/3nlv7aFz7Ii6zLA2NZ0YqVI3y0qiqJ7eBY0xLlvwXxNFPpOUBFunwZK3IO4ahI4AzyLWeEbyHt8AR1ZZi/pajnRWpCkuXY9j5sbjfLMuklOXY3ATmDKoCW3qlNdtW5XLykqiEJNBv5Q9r8lNuuAun7t6Fv54A7bPgKLlIeEG9JlhJYsjq2DuQOg59ebk4WTxiUks2v0Xy/eeYfm+s/RrVo2v1xxhVPcG3B+c5n5dSjlFVhLFCuAH4GdjzLFUxwthDWwPAJYbY6bmRMDZoS2KfC4y3OqOOrvXmk4b8hjsnONySeJWydu2Ji/wvqteBQaF+dHMr7ROr1UuISvTY7sAicBMETkpIn+KyGHgANAXGOOKSUIVAL5hMHQ1dHjT6pbaMB4qN7bGMFxU8ratz7SvRQlvT+5reAcbjlygz8T13PPpGi13rlxamiuajDExWMX/vhART6yKsTeMMZdyKTal0uZRCKqEgFdx8CoGBxbDhFbQ+1so7efs6G5y65hE85plGD5jG2N6BXHqSgyzNh2nYgmr1tWfJ69QrrgX5Yp7OTlqpf6m02NV3pR6TMK3FSx6HdZ/YXVF3fU2NB3itF31bjVh5SEaVvG5aeA6rVlP3catYe+paO4LvIPBLX2pf4dz9+5QBUuBWEehg9kFyJoxULnRzWMSu36w1l6cPwBVmsL9n0O5vLXl6eGzV5m6NpK5m6O4EZ9IM7/SPNuhNqG1dHaUynkFIlEk0xZFAWYM7JgNv//L2r+77b8g9FmXqhtlj8vX45m9+RjT1h7lmQ616N2kGjfiEolPSqKEt6ezw1P5VHb3o6gO1DbGLLFtZORhjHHZ0TdNFIro09Z+3Xt+gUqBVuuiYgNnR5VpCYlJGMDT3Y3Ja47wv8X76BlSlYGhvviWLers8FQ+k+WigCLyBPA98KXtUBVgnkOjU8rRilewBrZ7fQNXTsLEtrDsXatSbR7i4e6Wso938xpl6FS/ItM3HKXd/1bw+LTNrD10zskRqoLAnqKAEUBTYIMxJth2bKcxxmXfnmmLQt3k+gVY+CrsmAXl6lqtiyouu51Khs5cieG79Uf5bsMxapcvxuyhLQBITDK46y58KhuyU2Y81hgTl+pCHqRTA0opl1OkNPT4Eh6eC7FXYFJHWPxvawwjDypfwpvnO/mz9pX2/K9XIGAlj7DRy/h48T7OXIlxcoQqv7EnUawUkdeAwiJyFzAXmJ+zYSmVA+p0gqfWQaNHYe1nMD4Mjq51dlRZ5u3pTpVSRQC4HpdIQGUfPlt+kLD3l/Hc7Ah2Rum+LMox7Ol6cgMeAzph7Zm9CPjalWo8JdPpscpuh1fCLyPg0lFrN72Ob1qL9/K4yHPXbNNrj3MjPpF1r3aggm0xn1IZye6sp8JAteR9s12djlEou8Rdg6X/gQ0TwKcqdBsLNds7OyqHuBITz/pD5+lUvyIAr/20E98yRejdpBo+hXV6rbq97Mx66gZEAAttz4NE5BeHR6hUbitUFLqOhsGLrO1Yv+0OPz8NNy45O7JsK+HtmZIk4hKSOHz2Ku/9tpcWo5byxs+7OHz2qpMjVHmJPV1PW4D2wAqd9aTyrfgYWDkawj+FouXg3k/gzrudHZVD7T55mSnhkfwScZK4xCTG9gni/iAtda7+lp1ZT/HGmFtHxVxufEKpbPH0ho5vwRNLoWhZmNUXvn8MruWfdQr17/Dho56BhL/SnpEda9OqdjkAVu4/y4wNx7gRl+jkCJWrsqeuwW4ReRhwF5HawDNA3p0qolR67giGJ5ZD+BhY+QEcXg53fwj1e7jkft1ZUa64FyM7/l0Da8H2k8zdEsUHi/bycNNqiAhhtcrYVcRQFQz2tChGAPWBWGAGcBl4NieDUsqpPApBm5dh6CooWR2+HwyzHoErp5wdWY744KGGzB7SnGZ+pRm/8hDjVxxk8NRNKau+k8ukN6yilWwLKnvGKHoaY+ZmdMyV6BiFcpjEBKt8+fJ3wd0LurwHQY/km9bFrY6dv860dZF4ursxZ/NxeodU4dv1xxjfr1FKV5XKv7I8PVZEthpjGmV0zJVoolAOd/4Q/Dwcjq21ptDeNxZKVnN2VDkqeetWgDt8vOnfwpe+TatSskghJ0emckqmB7NFpKuIfAZUFpFPUz2mAgk5GKtSrqdMTRj4K9z9ERzbAF+0gI1fWVux5kPJW7cOb1eLYl4elCpaiPcX7qX5qKW8/tNO4hLy59etbi+9MYqTwGYgBtiS6vEL0DnnQ8s8EblPRCZevqylC1QOcHODpk9YZUCqNLHKmE+712pt5COpt259sbM/Ex9tzKnLMYzu0YD7Aytz+Ow1CnlYfzr2n47GBYs0KAezp+vJ0xgTn0vxOIR2PakcZwxETIeFr0FiLLT/NzR/ymW2X82OjLZuTUoyuLkJ56/G0mL0MqqWKsygMD96NKpMkUJ5a4ModbPsjFGEAW8B1bGm0wpgjDE1ciBOh9BEoXLNlVPw6/Ow7zeo3NgqYV6+rrOjyhVxCUn8tvMUk8OPsCPqMj6FPenTtCqPt6xBueJezg5PZUF2EsVe4DmsbqeUFTnGmPOODtJRNFGoXGWMtV/37y9DzBVo8y9oORLcC0ZNJWMMW45eZEp4JIt2/8Ufz7fBr2xRrsclUNjTHcmnM8Tyo+wkig3GmGY5FlkO0EShnOLaOStZ7PoBKjSA+8fBHUHOjipXnb8aS5liVmtiyDebOR0dy+AwX+5uUCllpz7lurJTwmO5iHwoIi1EpFHyIwdiVCpvK1oWHpoMvafDtTPwVXtY8rZVR6qASE4SAK3rlCP6RjzPzoqg5fvL+Hz5QS5ci0vnbOWq7GlRLL/NYWOMcdl6zNqiUE534yIs+jdEfAdFylrdUc2G/P35I6vgxFariyofS0oyrNx/lsnhR1h94BzPdqjNc3fVyfhE5RTZ2o8ir9FEoVzGwSXw0zC4dhbq3g/dx8OJLTB3IPScCn6tnR1hrtl/OpoyRQtRppgXi3b/xXfrjzIozJe2dcrjpnt9u4S0EoVdc9lE5B6sek8pW2UZY95xXHhK5VO1OsIz2+DHIbDnZziyEjDQ+7sClSQA6lT4ewfBmPhE9p+OZvDUzfiVLcrAUF8ealyFol46vdYV2bNx0QSgN1ZxQAF6Yk2VVUrZw6s49J0JgX0g5hLEXIZDyyGh4PbX3x9UmTX/as/YPkGUKOzJm7/sps/E9c4OS6XBnsHsUGPMo8BFY8zbQAtAOxmVyowjq+DAHxD2rLWb3pqP4ev2cPpPZ0fmNJ7ubtwfVJmfnw7jx6dCeaGT9WclJj6RF+ZsZ1PkBV317SLsaeclT9m4LiJ3AOeBSjkXklL5zJFVN49J1OoIsx6GS8dgYhvo8AY0f9oqEVJANapWKuXj/aejWbLnND9sjSKgcgkGh/lxb8M7UsqGqNxnz3d+voiUBD4EtgKRWPtSKKXscWLrzQPXfq2hzwxoOhRqd4LF/4Zp91mJQ9GwSknWvdqed7sHcCMukefnbCfs/WWcvlJwphm7mnRnPYmIG9DcGLPW9twL8L7N1qguRWc9qTzDGIiYAb//y3re9X0Iejjf7neRWUlJhtUHz7F87xnevK8eIsK8bSfwr1icupVKODu8fCc7K7O3GWOCcyyyDIjIA8A9QAlgkjFmcUbnaKJQec7FozDvSTgaDnfea+13UbRsxucVMHEJSbQYtZTz1+JoUaMMg1v60f7O8rjr9FqHyM7K7KUi8qBkoWCLiEwWkTMisuuW411EZJ+IHBSRV9K7hjFmnjHmCWAY1uwrpfKfUtVhwALo9F84sBi+aA77fnd2VC6nkIcbS19owytd7+To+Ws88c1m2n20gvCD55wdWr5mT4siGiiKtVlRDH9Xj82w3ScirYGrwDfGmADbMXdgP3AXEAVsAvoC7sCoWy4x2Bhzxnbe/4DpxpitGd1XWxQqTzu9G34cCqd3QnB/6DLKmmKrbpKQmMSi3aeZHH6EUT0aUKdCcSLPXcNNhGplijg7vDzJaSuzRcQXWJAqUbQA3jLGdLY9fxXAGHNrkkg+X4DRwB/GmCX23FMThcrzEmJhxSgIHws+VaH7l1C9hbOjcnkjZm5jwY6T3FW3AoNb+tHMr7RWr82E7HQ9ISKlRKSpiLROfmQjlsrA8VTPo2zH0jIC6Ag8JCLD0olxiIhsFpHNZ8+ezUZ4SrkADy/o+BYM+t0a2J7SFf54w0ogKk3/vqcuT7etxabIC/SZuJ67P13Dgh0nnR1WnmfPyuzHgVXAIuBt279v5WxYfzPGfGqMaWyMGWaMmZDO6yYaY0KMMSHlypXLrfCUylnVmsOwNdDoUat18VV7q2tK3VaFEt682Nmfda92YHSPBiQmJbH75BXAmkF1NloTbVbY06J4FmgCHDXGtAOCgUvZuOcJoGqq51Vsx7JN98xW+ZJXcej2KfSdDVfPwMS2VtJISszw1ILK29OdPk2rsWhka57tUBuAFfvPEDZ6Gc/PiWDXCf0bkRn2JIoYY0wMWOsojDF7Af9s3HMTUFtE/ESkENAH+CUb10thjJlvjBni4+PjiMsp5Vr8u8BT66xFen+8AVPvhYuRzo7KpYkI3p7WPua1yxenb9OqLNz1F/d+toZeX65j4a5TJCZpmZCM2JMoomwrs+cBf4jIz8BRey4uIjOBdYC/iESJyGPGmARgOFYX1h5gjjFG29JK2aNoWavy7APj4a+dMD4Mtn5rLdxT6apaughv3x/Aulc78O976nLi4g3+s2BPSj0pTRhpy9SsJxFpA/gAC40xLlf6UkTuA+6rVavWEwcOHHB2OErlrEvHYN5TELka/O+xFukV0/E5eyUkJhF18Qa+ZYsSm5BI509W0da/PANCffErW9TZ4TlFdlZmV7vdcWOMyxam0emxqsBISoL1X8DSd/4ey7jzHmdHledcuBbHf3/9k/nbT5KQZOhwZ3kGhfkRWrNMgZpem51EsRMwWAvtvAE/YJ8xpn5OBOoImihUgXP6T/hpiNUdFdwPOo8Cb62FlFlnomP4bv0xpq8/yvlrcXw/rAUhvqWdHVaucdiCOxFpBDxljHncUcE5miYKVSAlxMHK0bDmE/CpYlukF+rsqPKkmPhElu45w90NKiIijFmyn8QkQ7/m1alQwjvjC+RR2Vpwl5qthEYzh0TlYDo9VhVoHoWsvS0GLQRxgyl3w+L/00V6WeDt6c49DSuldDsdPX+dccsP0vL9ZYyctY0dUZecG2Aus6fr6flUT92ARkCZ5BIcrkhbFKrAi70Ki1+HLVOhfH3o8SVUbODsqPK0o+evMXVtJHM3R3E1NoEXO9VhePvazg7LobIzRvFmqqcJWBsX/ZC8tsIVaaJQymb/Ivh5ONy4CO1fh9BnwM3d2VHladEx8czdHEVYrbL4VyzO7pOXWXPgHH2aVMOniKezw8sWpxUFdAZNFEqlcu08LBgJe36Bai2sNRil/ZwdVb7x+fKDfLhoH4U93XmocRUGhvlSs1wxZ4eVJdlpUczHmvWU2mVgM/ClK7UsdB2FUmkwBnbMht9eApMEnd+z6kcVoKmfOenPk1eYEn6EnyNOEpeYRLfAO/i0r9P2e8uy7CSKsUA5YKbtUG/gClbyKGGM6e/gWLNNWxRKpeHScWsnvcjVUKerte6iWHlnR5VvnI2OZcaGY3i4C0+3q4Uxhp8jTtK5fkUKF3L9Lr/sJIpNxpgmtzsmIrtdcT2FJgql0pGUBBvGw5K3wauYtaK77n3Ojipf2hR5gZ4T1lGyiCcPN61G/xbVqeRT2NlhpSk702OLpV6dbfs4uQPO5cp4KKUy4OYGLZ6GoSuhRGWY3c8qBRJzxdmR5Tsh1UsxZ2gLmvuVYcLKQ7R6fzkjZm7j/NW8NWXZw47XvACsEZFDWKuz/YCnRKQoMC0ng8usVGMUzg5FKddXvi48vhRWvg9rPoYjq6H7ePBt6ezI8g0RoalfaZr6leb4hetMWxvJiv1nKe5tzY46eOYq1csUwdM900vacpVds55ExAu40/Z0nysNYN+Odj0plUnHN8JPQ+HCEWtmVKsXoHbHvz9/ZBWc2AotRzotxPwiKcng5ibEJybR6v3liED/FtXp26QapYoWcmps2VqZbYyJNcZstz1cOkkopbKgalMYuhpCBsGxtTCjF2yean3uyCqYOxAqN3JmhPmGm5s108xdhPd6BFCzXDE+WLiPFqOX8tpPO4k8d83JEf6TrqNQSt1s/2L4cQjEXITqLeHsHug5FfxaOzuyfGvfX9FMCT/CT9tOMPHRENrUKceNuES8PNxSEktu0AV3Sin7XTsPU++Gs3uhaHl4ZC7cEeTsqPK9C9fiKFnYEzc34b3f9rBkz2kGhfnxYKPKFClkz5By9mS560ks/UTkDdvzaiLSNCeCVEq5iDO74dpZqNfd+ndiO1jyFsRrz3NOKl20UEoLolG1khTz8uD/5u2i+XtLGfX7Hk5cuuGUuOxZRzEeSALaG2PqikgpYPGtaytcga7MVsoBksckkrub9v4KcwdBYiyUqQ33j4NqzZ0dZYFgjGHrsYtMXhPJ77tO0S3wDsb0ybkV39lZcLfVGNNIRLYZY4Jtx7YbYwJzKNZs064npbJhzRhr4Dr1mMSRVbBjDhxeAZejoNlQq6R5oYK5ZagznLh0g8REQ7UyRdj71xX+9f0OBoX5ceLSDYKrlSS0ZtmU1649dI4dUZcZ1qZmpu6RVqKwp9MrXkTcsdV7EpFyWC0MpVR+dLspsH6trUdstLWie8ME2Pe7VQKkRtvcjrBAqlzy7xXdF67FER2bwMjZEZQs7ElsQhKf9A6kS0Al1h46x/AZ2xj3sONaHva0KB7Bqu/UCGuB3UPAv40xcx0WhYNpi0KpHBYZDr+MgAuHrOKCnf4L3j7OjqpASUoyrDxwlslrjrD6wDkEeKptTWZuOs64h4NvamHYK1uznkTkTqAD1srspcaYPZmOIBdpolAqF8TfgOXvwbpxUKwC3PsJ+Hd1dlQF0oHT0fxv8X4W7v6LZ9rX4vlO/lm6TnZmPdUEjhhjPgd2AXeJSMksRaGUyj88C0On/8DjS6BwKZjZB3543Jpaq3LV2auxbIy8wDPta/HdhmOsPXTOode3Z2X2D0CiiNQCvgSqAjMcGoVSKu+q3BiGrIQ2r8Dun+DzprDrR2sPDJXjUo9JPN/Jn3EPBzN8xjaHJgt7EkWSMSYB6AGMM8a8BFRyWAQOJCL3icjEy5cvOzsUpQoWj0LQ7lUYugpKVoXvB1lVaaP/cnZk+d6OqMs3jUmE1izLuIeD2RHluL+D9gxmbwDGAK8D9xljjojILmNMgMOicDAdo1DKiRITYP3n1viFh5e1m17QI7qbXh6QnaKAg4AWwLu2JOEHfOvoAJVS+YS7B4Q9C8PCoXw9+Plp+K4HXDrm7MhUFmmtJ6VUzklKgs2T4I83rRZFx7cg5DFr8yTlcrIz66m2iHwvIn+KyOHkR86EqZTKV9zcoOkT8NQ6qNIEfnsRpt4D5w46OzKVCfak9SnAeCABaAd8A3yXk0EppfKZUtWh/09w/+dWwcEJYRA+1hrPUC7PnkRR2BizFKub6qgx5i3gnpwNSymV74hAcD94agPU7AB/vAGTOsLp3c6OTGXAnkQRKyJuwAERGS4i3YFiORyXUiq/KlEJ+kyHhybDpePwZRtYMRoS4pwdmUqDPYniWaAI8AzQGOgHDMjJoJRS+ZwIBDwIT2+E+g/AilEwsa21L7dyOTrrSSnlfPt+hwXPwdXTEDoC2r5qlQhRuSo7s57+SF3bSURKicgiB8fnELoyW6k8yr8rPLXeGsMIHwvjw+DoWmdHpWzs6Xoqa4y5lPzEGHMRKJ9jEWWDMWa+MWaIj4+WO1YqzylcErp9Bv3nQVI8TOkKv75o7YGhnMquWk8iUi35iYhUx7aJkVJKOVzNdvDkOmg2DDZ9DV+EwqFlzo6qQLMnUbwOrBGRb0XkO2AV8GrOhqWUKtC8ikHX92HwQqte1LfdYd7TcOOisyMrkDJMFMaYhVi7280GZgGNjTEuOUahlMpnqjWHYWug5XOwfSZ83hz2/ursqAocuwquGGPOGWMWACHGGMfuiKGUUunx9LZqRD2xFIqWhVkPw9xBcE3/FOWWzFbm6pYjUSilVEbuCIYnlkO712HPfBjXBHZ+rxsk5YLMJgotKK+Uch6PQtDmZRi2Gkr7wQ+Pwcy+cOWksyPL1+xZRxGW6mnj2xxTSqncVb4uPPYHdHoXDq+wxi62TNPWRQ6xp0XxWfIHxpikW48ppZRTuLlD6HB4MhwqNoD5z8C3D8DFSGdHlu94pPUJEWkBhALlROT5VJ8qAbjndGBKKWWXMjVhwHzYMsWqSPtFC+jwJjQdohskOUh638VCWFViPYDiqR5XgIdyPjSllLKTmxs0ecwqA1I9FBb+y1rZfe6AsyPLFzIsCigi1Y0xR20fuwHFjDFXciO4rNKigEoVYMbA9lmw8BWIvwFtX4HQZ6y9vFW6slwUEBglIiVEpCiwC/hTRF5yeIRKKeUIIhDU1yphXqczLH0bvm4Pf+10dmR5lj2Jop6tBfEA8DvgB/TPyaBSE5G6IjLBtm/3k7l1X6VUHle8AvT+FnpOs6bPTmwLy96FhFhnR5bn2JMoPEXEEytR/GKMicfOooAiMllEzojIrluOdxGRfSJyUEReSe8axpg9xphhQC9Ap+UqpTKn/gNW6yLgQVj1AXzZGqK0azoz7EkUE4BIoCiwylY91t4xiqlAl9QHRMQd+BzoCtQD+opIPRFpICILbnmUt53TDfgV+M3O+yql1N+KlIYeE+HhuVbZ8kl3waLXIe66syPLE9IdzLYNXj9kjJmT6pgA7saYBLtuIOILLDDGBNietwDeMsZ0tj1/FcAYM8qOa/1qjLknjc8NAYYAVKtWrfHRo0ftCU8pVdDEXLGm0W6ZAqVrWHtg+LZ0dlQuIUuD2bYFdi/fcszYmyTSUBk4nup5lO3YbYlIWxH5VES+JJ0WhTFmojEmxBgTUq5cuWyEp5TK17xLwH1jrLUXJgmm3gMLnrcSiLote+aLLRGRF7HKjF9LPmiMuZBjUaVijFkBrMiNeymlChC/1vDkWmuAe/0XsH8R3DcWand0dmQux54xit7A01gbFm2xPbIzEnQCqJrqeRXbsWzTPbOVUplSqCh0eQ8eW2x9PP1B+OlJuJ4r74PzDHs2LvK7zaNGNu65CagtIn4iUgjoA/ySjeuljlX3zFZKZV7VplZF2lYvwo7Z8Hkz+NMhf5byBXuqx3qKyDO2dQzfi8hw23TZDInITGAd4C8iUSLymG18YziwCNgDzDHG7M7OF6GUUtnm4QUd/g+GrIDiFWFOf5jzKFw94+zInM6eEh5fA57ANNuh/kCiMebxHI4t00TkPuC+WrVqPXHggNZ4UUplUWI8rP0UVoy2dU+9Dw17Wau+87G0Zj3Zkyi2G2MCMzrmSrTWk1LKIc7ug5+HQ9RGqN0J7h0DPmlO0szzslPrKVFEaqa6UA0g0ZHBKaWUSyrnD4MXQudRELkGvmgOm6cUuA2S7EkULwHLRWSFiKwElgEv5GxYWaOznpRSDufmDi2esqbS3hEEC0bCtPvgwmFnR5Zr0ux6EpGRwFpgK9ZGRf62T+0zxrh0VS3telJK5QhjYOs0WPx/1jhGh/+DZsOsZJIPZKXrqQowBjgDLMaaxloNq+aTUkoVPCLQeKC1QZJfa1j0GkzubI1l5GNpJgpjzIvGmFCgIvAqcAEYBOwSkT9zKT6llHI9PpXh4dnQ4ys4fxAmtIRVH1mtjHzInjGKwlj7ZPvYHieBDTkZVFbpGIVSKteIWFNmn94I/nfDsv/AV+3g1HZnR+Zw6Y1RTATqA9FYiWE9sN4YczH3wssaHaNQSuW6PfPh1xfg2jloORJavwye3s6OKlOyMkZRDfAC/sKqxRQFXMqR6JRSKq+rex88vQEC+8Dq/8GXreD4RmdH5RDpjVF0AZoAH9kOvQBsEpHFIvJ2bgSnlFJ5SuFS8MAX0O8HiL8BkzrBwlch7lrG57qwjPajMMaYXVj7QPwOhAM1gWdzITallMqbanWEp9ZBk8esEubjQ+HIKmdHlWVpJgpbIcBZInIMWAncC+wFegClcym+TNHBbKWUy/AqDvf8Dwb+CuJmLdKb/yzE5L2/T+kNZn+M1YJYa4w5latRZZMOZiulXErcdVjxHqz7HIpVtHbYq9PZ2VH9Q6YHs40xzxtjfshrSUIppVxOoSLQ6b/w2BLw9oEZveDHIXlmgyR71lEopZRyhCqNYehKaPMv2PUDfN4Uds9zdlQZ0kShlFK5ycML2r0GQ1ZCicowdwDM7gfRp50dWZryVaLQwWylVJ5RMQAeXwod34L9i63WRcQMlyxhnq8She6ZrZTKU9w9oOVz8GQ4lLsT5j0J0x+CS8edHdlN8lWiUEqpPKlsbRj0O3T9EI6uszZI2vQ1JCU5OzJAE4VSSrkGNzdoNsRaqFclxKobNe0+OH/I2ZFpolBKKZdSqjr0nwfdxsFfO2F8GKz9DJKctwO1JgqllHI1ItCov1VksGY7WPxvmHQXnNnjlHA0USillKsqUQn6zIAHJ8HFSJjQClZ+kOsbJOWrRKHTY5VS+Y4INHjI2iCp3v2w/F2Y2BZObsu1EPJVotDpsUqpfKtoWXhoEvSZaW2O9FUHWPIWxMfk+K3zVaJQSql87867rbGLoIdhzSfwST3YMOHm1xxZBWvGOOyWmiiUUiqvKVwS7h8H/X8CcYff/wWzHoHYq1aSmDsQKjdy2O08HHYlpZRSuatme3hmG/w0FPYugI/rWntf9P4W/Fo77DbaolBKqbzMqxj0mQ6BD0PsFWg80KFJAjRRKKVU3ndkFRxYBK1fhm3fOnzbVU0USimVlyWPSfScCu1ft/6dO9ChyUIThVJK5WUntlrJIbm7ya+19fzEVofdQgezlVIqL2s58p/H/FrrYHZadGW2Uko5Xr5KFLoyWymlHC9fJQqllFKOp4lCKaVUujRRKKWUSpcYY5wdg8OJyFngqLPjuEVZ4Jyzg7BTXooV8la8eSlWyFvx5qVYwTXjrW6MKXfrwXyZKFyRiGw2xoQ4Ow575KVYIW/Fm5dihbwVb16KFfJWvNr1pJRSKl2aKJRSSqVLE0XumejsADIhL8UKeSvevBQr5K1481KskIfi1TEKpZRS6dIWhVJKqXRposhBIlJVRJaLyJ8isltEnnV2TPYQEXcR2SYiC5wdS3pEpKSIfC8ie0Vkj4i0cHZM6RGR52w/B7tEZKaIeDs7ptREZLKInBGRXamOlRaRP0TkgO3fUs6MMVkasX5o+1nYISI/iUhJJ4aY4naxpvrcCyJiRKSsM2KzlyaKnJUAvGCMqQc0B54WkXpOjskezwJ7nB2EHcYCC40xdwKBuHDMIlIZeAYIMcYEAO5AH+dG9Q9TgS63HHsFWGqMqQ0stT13BVP5Z6x/AAHGmIbAfuDV3A4qDVP5Z6yISFWgE3AstwPKLE0UOcgYc8oYs9X2cTTWH7LKzo0qfSJSBbgH+NrZsaRHRHyA1sAkAGNMnDHmklODypgHUFhEPIAiwEknx3MTY8wq4MIth+8Hptk+ngY8kJsxpeV2sRpjFhtjEmxP1wNVcj2w20jj+wrwCfAy4PIDxZoocomI+ALBwAYnh5KRMVg/vElOjiMjfsBZYIqtm+xrESnq7KDSYow5AXyE9e7xFHDZGLPYuVHZpYIx5pTt47+ACs4MJhMGA787O4i0iMj9wAljzHZnx2IPTRS5QESKAT8AI40xV5wdT1pE5F7gjDFmi7NjsYMH0AgYb4wJBq7hOt0i/2Dr278fK8HdARQVkX7OjSpzjDVF0uXf/YrI61jdvtOdHcvtiEgR4DXgDWfHYi9NFDlMRDyxksR0Y8yPzo4nA2FANxGJBGYB7UXkO+eGlKYoIMoYk9xC+x4rcbiqjsARY8xZY0w88CMQ6uSY7HFaRCoB2P494+R40iUiA4F7gUeM6879r4n1hmG77XetCrBVRCo6Nap0aKLIQSIiWH3oe4wxHzs7nowYY141xlQxxvhiDbQuM8a45LteY8xfwHER8bcd6gD86cSQMnIMaC4iRWw/Fx1w4cH3VH4BBtg+HgD87MRY0iUiXbC6TbsZY647O560GGN2GmPKG2N8bb9rUUAj28+0S9JEkbPCgP5Y78wjbI+7nR1UPjICmC4iO4Ag4D3nhpM2W8vne2ArsBPrd8+lVuaKyExgHeAvIlEi8hgwGrhLRA5gtYpGOzPGZGnEOg4oDvxh+12b4NQgbdKINU/RldlKKaXSpS0KpZRS6dJEoZRSKl2aKJRSSqVLE4VSSql0aaJQSimVLk0UKs8Tkat2vGakbUWso+75QOoCjyLyjoh0dNT1c4qt4u5Tzo5D5S2aKFRBMRKrEJ/dRMQ9nU8/AKQkCmPMG8aYJVmKLHeVBDRRqEzRRKHyDRFpKyIrUu1RMV0sz2DVV1ouIsttr+0kIutEZKuIzLXV40JEIkXkfRHZCvQUkSdEZJOIbBeRH2wrq0OBbsCHtoVdNUVkqog8ZLtGB1uhwp22vQi8Ul37bds9d4rInbf5GtxF5CPbnhU7RGSEHdcsa/s4RERW2D5+y/a6FSJy2PY9AGvBXE1b3B/ecm9fsfb1+EqsfTMWi0hhx/4vqbxIE4XKb4KxWg/1gBpAmDHmU6yS3u2MMe1sf1j/DXQ0xjQCNgPPp7rGeWNMI2PMLOBHY0wTY0zyfhePGWPWYpW2eMkYE2SMOZR8olibEU0FehtjGmAVL3wy1bXP2e45HnjxNvEPAXyBINu+CtPtuGZa7gQ6A02BN211x14BDtnifuk259QGPjfG1AcuAQ/acR+Vz2miUPnNRmNMlDEmCYjA+qN7q+ZYiSRcRCKwahhVT/X52ak+DhCR1SKyE3gEqJ/B/f2xiv/ttz2fhrVvRrLkwpBb0oitI/Bl8r4KxpgLdlwzLb8aY2KNMeewivnZUyL8iDEmIoMYVQHj4ewAlHKw2FQfJ3L7n3EB/jDG9E3jGtdSfTwVeMAYs91WmbStg+JLK7bMSuDvN3y3bq1qz/fiVreeo11PSlsUqsCIxioYB9buZ2EiUgtARIqKSJ00zisOnLJ12zySxvVS2wf4Jl8bqyjkykzE+QcwVKxd8BCR0hlcMxJobPvYnm6itOJWKk2aKFRBMRFYKCLLjTFngYHATFvl2XVY/fm3839YuxKGA3tTHZ8FvGQbYK6ZfNAYEwMMAubauquSgMxUMf0aqyT5DhHZDjycwTXfBsaKyGasFkC6jDHnsbrcdt06mK1UWrR6rFJKqXRpi0IppVS6NFEopZRKlyYKpZRS6dJEoZRSKl2aKJRSSqVLE4VSSql0aaJQSimVLk0USiml0vX/rd9Dk5n0rzEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot theoretical and PEPit (numerical) worst-case performance bounds as functions of the iteration count\n", "\n", "plt.plot(n_list, theoretical_taus, '--x', label='Theoretical upper bound')\n", "plt.plot(n_list, pepit_taus, '-x', label='PEPit worst-case bound')\n", "\n", "plt.semilogy()\n", "plt.legend()\n", "plt.xlabel('Interation count n')\n", "plt.ylabel('Worst-case guarantee (log scale)')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
↩ Back to TOC
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.3 Example 3 : An accelerated Douglas-Rachford splitting" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We consider the following composite convex minimization problem\n", "\\begin{equation}\n", " F_\\star \\triangleq \\min_x \\{F(x) \\equiv f_1(x) + f_2(x)\\},\n", "\\end{equation}\n", "where $f_1$ is closed proper and convex, and $f_2$ is $\\mu$-strongly convex and $L$-smooth. We assume access to the proximal operator of both $f_1$ and $f_2$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The package PEPit allows to compute a worst-case guarantee for **accelerated Douglas-Rachford**. That is, it computes the smallest possible $\\tau(n, L, \\mu, \\alpha)$ such that the following guarantee holds (for all initilization of the algorithm, and all functions satisfying the assumptions):\n", "\\begin{equation}\n", "F(y_n) - F(x_\\star) \\leqslant \\tau(n, L, \\mu, \\alpha) \\| w_0 - w_\\star \\|^2,\n", "\\end{equation}\n", "where $\\alpha$ is a parameter of the method, and where $y_n$ is the output of the accelerated Douglas-Rachford method, where $x_\\star$ is a minimizer of $F$, and $w_\\star$ defined such that\n", "\\begin{equation}\n", "x_\\star = \\text{prox}_{\\alpha f_2}(w_\\star).\n", "\\end{equation}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The accelerated Douglas-Rachford method under consideration is be described as follows, for $t \\in \\{0,1, \\ldots, n-1\\}$\n", "\\begin{equation}\n", " \\begin{aligned}\n", " x_t &= \\mathrm{prox}_{\\alpha f_2}(u_t), \\\\\n", " y_t &= \\mathrm{prox}_{\\alpha f_1}(2x_t - u_t), \\\\\n", " w_{t+1} &= u_t +\\theta (y_t - x_t), \\\\\n", " u_{t+1} &= \\left\\{\\begin{array}{ll} w_{t+1}+\\frac{t-1}{t+2}(w_{t+1}-w_t)\\, & \\mbox{if } t \\geqslant 1,\\\\\n", " w_{t+1} & \\mbox{otherwise,} \\end{array} \\right.\n", " \\end{aligned}\n", "\\end{equation}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Theoretical guarantee" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is no known worst-case guarantee for this method beyond quadratic minimization. When the objective function is a quadratic, a worst-case **upper** bound is provided by\n", "\\begin{equation}\n", " F(y_n) - F_\\star \\leqslant \\frac{2\\|w_0 - w_\\star\\|^2_2}{\\alpha \\theta(n+3)^2},\n", "\\end{equation}\n", "when $\\theta = \\frac{1 - \\alpha L}{1 + \\alpha L}$ and $\\alpha < \\frac{1}{L}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot worst-case guarantees as functions of the iteration count" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the parameters\n", "L = 1 # smoothness parameter\n", "mu = 0.1 # strong convexity parameter\n", "gamma = 1 / L # step-size\n", "alpha = 0.9\n", "\n", "# Set a list of iteration counter to test\n", "n_list = np.array([1, 2, 4, 6, 8, 10, 15])\n", "\n", "\n", "# Compute numerical and theoretical (analytical) worst-case guarantees for each iteration count\n", "pepit_taus = list()\n", "theoretical_taus = list()\n", "for n in n_list:\n", " pepit_tau, theoretical_tau = wc_accelerated_douglas_rachford_splitting(mu=mu,\n", " L=L,\n", " alpha=alpha,\n", " n=n,\n", " verbose=verbose,\n", " )\n", " pepit_taus.append(pepit_tau)\n", " theoretical_taus.append(theoretical_tau)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEKCAYAAAAMzhLIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABFeUlEQVR4nO3dd3hUZfbA8e9JhwAJEHpLoZNGKAFCRxAXwVXAAqhYEARsu2LZde3u6k9XXATFCqigCKsoujaQDgKhSseE3gMktCSkvL8/7mRIIGVSJpNyPs8zT+beuXPvmUkyZ94uxhiUUkqpvLi5OgCllFJlmyYKpZRS+dJEoZRSKl+aKJRSSuVLE4VSSql8aaJQSimVLw9XB+AMAQEBJjAw0NVhKKVUubFhw4YEY0yd3B6rkIkiMDCQ2NhYV4ehlFLlhogcyOsxrXpSSimVrwqVKERksIi8n5SU5OpQlFKqwqhQicIYs9AY84Cfn5+rQ1FKqQqjQiUKpZRSJU8TBTB9WRyr4xJy7Fsdl8D0ZXEuikgppcoOTRRAeGM/Js7ZZE8Wq+MSmDhnE+GNtQpLKaUqZPfYwuoWEsDUEe2ZOGcTN4Y34Lutx5g6oj3dQgJcHZpSSrmclihsuoUEcFNEQz5Zc4CMzEyW7DrJun1nyMjU9TqUUpWbJgqb1XEJLNh8hN6t6nDpcgYfr9rHre+todMri/gt/rSrw1NKKZfRqieutElMGxlFt5AAVsclMGH2Ru7s2owDpy8RXMcXgHmxh/hp+wkGtK1H3zZ1Cajm7eLIlVLK+TRRAFsPJ+Vok+gWEsC0kVFsPZzEf25vbz/uckYmO44msWjnCUSgQ9OaDGhXjzE9ghERV4WvlFJOJRVxzeyOHTsaZ831ZIxh+9Fz/LLjBL/sOIGnhxvfTIgB4MvYQ4TUqUb7Jv64uWniUEqVHyKywRjTMbfHtERRSCJCaCM/Qhv58Vj/liRfzgAgNT2DF77dzsXLGdSp7s11berSv209uoUE4OPp7uKolVKq6LQxu5iqeFlJwNvDndVP9eM/t0fSOagWC7cc496ZsXywPB6wEsnZi5ddGapSShWJlihKkF9VT26KbMRNkY1ITc/gt/gzhNgawpfuPsX42Rvp2Kwm/dvWY0Db+jStXdXFESulVMG0jaKU7E+4yPwNh/llxwl2nzgPQKt61Zk9Jlp7TymlXE7bKMqAwABfHr++FY9f34qDpy/x847jbDqYSG1fLwBe/2kXiZfSGNCuPl2Ca+Htoe0aSqmyQROFCzStXZX7ewTn2Hf2UhoLNh1h9tqDVPP2oFerOtwc2Yjr2tZzUZRKKWXRxuwy4p83h7HxH/35eHRHBkc0YG38GZbuOQlAZqZh9toDHElMdnGUSqnKSNsoyqjMTMPFy+lU9/Fk25Ekbnx7JQBtG9Sgf9t69G9bj3YNa+hAP6VUicivjUJLFGWUm5tQ3ccTgNBGfvz61148fUNrqnq5M+XXvdz49kpW7LWmRb+Qmk5aRqYrw1VKVWDaRlFOBNepxthe1RjbK4SEC6n8uvMknYNqAfDesjhmrd5Pn9bWIL9eLevYk4xSShVXhUoUIjIYGNy8eXNXh+JUAdW8ubVTE/t2t5AAjielsHjXSb7ZfBRPd6Ff63q8OypKq6aUUsVWoRKFMWYhsLBjx45jXB1LaeoaUpuuIbXJyDRsPHiWX3ac4HJ6pj1J/GXuZoICfOnfrh6t6lXX5KGUKpQKlSgqO3c3oVNgLToF1rLvS0nLYN/pi3y16Qj//mUPTWpVoX+b+gzr0Ji2DWu4MFqlVHnhUKIQkZpAQyAZ2G+M0ZbTcsLH052vx8dw8lwKi3ae5Jcdx/ls7QFa1a9G24Y1OHU+lQ0HztCjRR18vfV7g1LqWnl+MoiIHzABuAPwAk4BPkA9EfkNeMcYs6RUolTFVreGDyOimzIiuikXU9Nxs1U//bT9OM8s2IaXhxvdmwfQv209+rWpS93qPi6OWClVVuT3FXI+8AnQwxiTmP0BEekA3CkiwcaYj5wYn3KC7CWH2zs1IbiOr319jV93ncRNYMMz/anp68XF1HSqerlru4ZSlZgOuFN2xhh2HT/PlkOJ3N65KQCjZ6zjwOlL9kF+UU1r4q6LMilV4RRrwJ1YRonIs7btpiLSuaSDVK4nIrRpUMOeJAAGtqtP45pVmLFqH8Onr6HzK4uYtuQPAKYvi2N1XEKOc6yOS2D6srhSjVsp5VyOtF6+A2QCfYEXgfPAf4FOToxLlRG3d27K7Z2bci4ljWW7T7Fo5wmq2hZralmvGnd/vI57YoKY0Kc5248mMXHOJqaOaF/AWZVS5YkjiSLaGBMlIpsAjDFnRcTLyXGpMqaGjyeDIxoyOKKhfV8DvyrU8PHg/eXxzFi1D3c34d/DI+gWEuDCSJVSJc2RuZ7SRMQdMAAiUgerhKEquTYNahD7TH9u7diYtAxDSlomj83dwsHTl1wdmlKqBDmSKKYAXwN1ReQVYCXwT6dGpcqNNfGnWbTzJA/3bY5fFU+GdWxkX+L183UH2XTwrIsjVEoVV4FVT8aY2SKyAegHCPBnY8xOp0emyrzVcQn2NoluIQF0CanNxDmbuDE8gQ7NajL5lz2cPJ9Kl+BajOsVQq+WdbSbrVLlUJ7dY0WkVq4P2BhjzjglohKg3WNLx/RlcYQ39svRJrE6LoGth5MY1yuEC6npfLHuIB+u2Mfxcym0aVCDf94cSvumNV0YtVIqN/l1j80vUezDapfI/hUwa9sYY4JzfWIZoImibLmcnsk3m4/w4Yp9TL+zA0EBvhxPSsGviidVvHRtcKXKgvwSRZ5VT8aYIOeFpCoTLw83hndswrAOje1VT//4ZhsbDpxldLdA7uraDP+q2pFOqbKqMJMCtsCa6wkAY8xyZwWlKqbs7RP3dw8iPSOTN3/Zw/RlcdzRuSn3dQ+ioX8VF0aolMpNgYlCRO4HHgEaA5uBLsAarAF4ShVJdHBtooNrs+v4Od5bFs/M1fvx9XLnLwNauTo0pdRVHOke+wjWKOwDxpg+QHsg0ZlBqcqjdf0aTL4tkmWTenNvd6u28+ftx7l/ViwbDpTZ/hJKVSqOVD2lGGNSRAQR8TbG7BIR/dqnSlTjmlXt95OS04g9cIah756gU2BNxvUKoU+rurjpZIRKuYQjJYrDIuIPLAB+EZFvgAPODEpVbsM7NmH1U3159sa2HDmbzH2zYnngU+3FppSrFGqacRHpBfgBPxpjLjstqmLS7rEVR1pGJgu3HMXbw51B4Q1ISctgXuwhhnZoTFUvXZFPqZJS3GnGu4hIdQBjzDJgKVY7hVJO5+nuxi1RjRkU3gCAJbtO8o9vthPz6q9M/mUPZy6W2e8rSlUYjlQ9vQtcyLZ9wbZPqVJ3Q1gD/vtgVzo0q8V/Fu8l5tVfef7b7aSmZ7g6NKUqLEfK7mKy1U8ZYzJFRMv8ymU6NKvFh3fXYu+J80xfFs+2I0l4uVvfeRIupBJQzdvFESpVsTjygR8vIg9zpRQxHoh3XkhKOaZFver8+9YIMjINIsKZi5fp8doS+ySEnYNq6SSESpUAR6qexgHdgCPAYSAaeMCZQSlVGFlreHu6C+N7h7D1cBK3vf8bt7y7mp+3Hyczs+KtC69UaSpUr6fyQns9VW7JlzOYt+EQ7y+P5/DZZBb9pRfN61ZzdVhKlWnF7fX0fyJSQ0Q8RWSxiJwSkVElH6ZSJaOKlzt3dQ1k6eO9+XxMF3uSeGHhdj5YHs+F1HQXR6hU+eJI1dMAY8w54EZgP9AcmOTMoJQqCR7ubnQNqQ1AekYme09c4JX/7aTbvxbzxk+7SbiQ6uIIlSofHEkUWQ3eg4B5xpgkJ8ajlFN4uLvx2f3RLJgQQ7eQAKYt/YOYV3/lp+3HXR2aUmWeI72evhORXUAy8KCI1AFSnBuWUs4R2cSf6Xd2IO7UBT5cEU/7Jv4A/H44CREIbeTn2gCVKoMcasy2LYuaZIzJEBFfoLoxpsx+FdPGbFVYo2esY+nuU/RoEcCDvULoGlJbu9aqSqVYjdlgrY9tjMmw3b9YlpOEUkXxn9vbM+n6Vuw8dp4RH67lz9NWsXT3SVeHpVSZ4FCicCUR8RWRWSLygYiMdHU8qmLyq+LJhD7NWflkH165OZTE5DT+OGnNXJOWkalThKhKzSWJQkQ+FpGTIrLtqv0DRWS3iPwhIk/Zdt8CzDfGjAGGlHqwqlLx8XRnZHQzfv1rb+7qGgjA15uO0OO1JUxfFse5lDTXBqiUCzgyjiIql1tIMed7mgkMvOo67sA04AagLXCHiLTFWoL1kO0w/VqnSoW7m+DlYf17hNTxpWW96rz6wy5i/vUrr/6wi5PntD+Hqjwc+bB/B4gCtgIChALbAT8RedAY83NhL2qMWS4igVft7gz8YYyJBxCRL4CbsKYNyVqvu8xXlamKp0OzWnx2fzS/H05i+rI43l8ex2/xp1kwIcbVoSlVKhz54D0KtDfGdDTGdMBaiyIe6A/8XwnG0ogrJQewEkQj4CtgqIi8CyzM68ki8oCIxIpI7KlTp0owLKUsYY39mDYyil//2pvnBrcFrGVbH5u7ma2HE10bnFJO5EiJoqUxZnvWhjFmh4i0NsbEl0b3QWPMReAeB457H3gfrO6xzo5LVV6BAb4E4gvAzmPnWLTzBF9vOkJM89qM6xVC9+YB2rVWVSiOlCi2i8i7ItLLdnsH2CEi3kBJtuwdAZpk225s26dUmdUluDarn+rL0ze0Zu+JC9z50ToGT13Jpcs6n5SqOBwpUYzGWoPiUdv2KuBxrCTRpwRjWQ+0EJEgrARxOzCiBM+vlFNU9/FkbK8QRscEsmDTEX4/kmRfz3vVHwl0aFYTH093F0epVNEVmCiMMcki8jbwM2CA3caYrJLEhbyfmTcR+RzoDQSIyGHgOWPMRyIyEfgJcAc+zl7lpVRZ5+3hzm2dmnJbJ2v7xLkU7vp4HTWrenJPTBCjujTDr4qna4NUqggKnMJDRHoDs7BmjhWs6qG7jTHLnRxboYnIYGBw8+bNx+zdu9fV4ahKzhjDb/FnmL4sjmV7TuHr5c6I6KZU8XSnS0htuoUE2I9dHZfA1sNJjOsV4sKIVWVW3Ck8/o011XgvY0xP4HpgckkGWFKMMQuNMQ/4+enEbsr1RISuIbWZdW9nvn+4O/3a1OOTNQdoUa86E+dsYsVeq3fe6rgEJs7ZRHhj/btVZZMjbRSexpjdWRvGmD0iouVnpQqhXUM/ptzRnsRLl/Gv6kXtal7c9fE6mtepxvFzKbwzMipHCUOpssSREkWsiHwoIr1ttw8AnZpVqSLwr+oFQJeg2oQ38mPX8fNcSk1n17HzpGdkujg6pXLnSKJ4ENgBPGy77bDtU0oV0W/7TrP/9CXu7NIUA7z43Q4GTVlJ3Kki9Q9Ryqkc6fWUCrxpuymliimrTWLqiPZ0CwnghtAGPPDpBjJNJnWrewOQmWlwc9NBe6psyDNRiMjvWN1hc2WMCXdKREpVcFsPJ9mTBEC35gG8f1cHth5OorqPJ2kZmQybvoYbQutzb0yQfXJCpVwlz+6xItIsvycaYw44JaJi0O6xqiI4e/Eyk+ZvZdHOEwTX8eX5we3o2bKOq8NSFVx+3WPzSxRiChhk4cgxrqBLoaqKYMmuk7ywcDv7T19iYLv6vDYsXAfsKacp6jiKJSLykIg0vepkXiLSV0RmAXeXZKBKqSv6tK7LT4/1ZNL1rUi4kEo17+IsAaNU0eWXKAZiLRT0uYgcFZEdIhIP7AXuAN4yxswshRiVqrS8PdyZ0Kc5X47tirubkHQpjT9PW8WiHSdcHZqqRPL8imKMScFatOgd2wC7ACDZGJNYSrEppWyyekCdOJ/ChdR07v8klr6t6/LsjW0JDPB1cXSqonOoO4UxJs0Yc0yThFKu1bJedX54pAd//1Mb1safZsDk5bzx024yMstcU6GqQLTSU6lyxtPdjTE9g7kpsiH/+mEXu46fx13HXCgnqlAdtEVksIi8n5SU5OpQlHK6ujV8mHxbJO+OigJgf8JF7pmxjj9OnndxZKqicShRiEgzEbnOdr+KiFR3blhFo7PHqsrI0936N45PuMCGA2cZ+NYK/vm/nVxI1VX2VMkoMFGIyBhgPvCebVdjYIETY1JKFUHf1vVY8nhvhkY15v3l8fR9YynfbNbVhFXxOVKimADEAOcAjDF7gbrODEopVTS1q3nz2rBwFkyIob6fD5sPJbo6JFUBONKYnWqMuSxiNZaJiAf5zAGllHK9yCb+LBgfw2Xb1OW/xZ/mx23Heax/Sx3drQrNkRLFMhH5G1BFRPoD84CFzg1LKVVcbm6Cj6c7AFsOJfLJmv30fWMpX8YeIlO706pCcCRRPAWcAn4HxgL/A55xZlBKqZI1tlcI307sTmCAL0/M38ot767m98PaO1A5xpH1KDJF5DNgefYlUZVS5UtoIz/mj+vK15uO8M//7WL9/jOE6TrdygEFJgoRGQK8DngBQSISCbxojBni5NgKLds0464ORakySUS4Jaox17WtR1VbtdTCLUdJTE5jROemOnBP5cqRqqfngM5AIoAxZjMQ5LyQik7HUSjlmBo+nnjYxl/8vOME/1iwjSFTV7LhwBkXR6bKIkcSRZox5urKTG0JU6qCmHJ7JG/f0Z7TFy4z9N01/PXLLZw6n+rqsFQZ4kii2C4iIwB3EWkhIm8Dq50cl1KqlIgIgyMasvivvRjXK4Rvtxxh08Gzrg5LlSF5rnBnP0CkKvB3YIBt10/AS8aYMvuVQ1e4U6rojiQm09DPBxHh83UHCQrwpUtwbVeHpZwsvxXuHBlwN8gY83esZJF1wuFY4ymUUhVMI/8qAKRnZPLhinjiTl2kTYPqTOzTnEHhDe3HrY5LYOvhJMb1CnFVqKqUOFL19LSD+5RSFYiHuxvfPdSDh/u1YO+JC0yYs4mnv9rK5fRMVsclMHHOJsK1e22lkGeJQkRuAP4ENBKRKdkeqgHotJRKVQJVvNz5S/+WDItqzKNzN/P5ukMA/LT9BFNHtKdbSICLI1SlIb8SxVEgFkgBNmS7fQtc7/zQlFJlRdPaVflqfDdGRjfl83WHGBXdlHPJaaSkZbg6NFUK8lszewuwRUTmGGPSSjEmpVQZtDougR+2Hefhvs2ZteYASclphNTx5fXhEUQ1renq8JQTOdJG0VlEfhGRPSISLyL7RCTe6ZEVga5wp5RzZLVJTB3Rnr8MaMW7o6Ko7uNB4qXLDHt3Nf/6YaeWLiowR7rH7gIew6p2sv8lGGNOOze0otPusUqVrOnL4ghv7JejTWJ1XALr953hWFIKX6w/RERjP74eH4ObTgNSLhW3e2ySMeaHEo5JKVWO5NYFtltIgD1x3BDWgDMXU3FzE4wxpGUYvDwcWmlZlQOOJIolIvI68BVgH2RnjNnotKiUUuVKr5Z17Pf/u/EI7y2L443hEUQ08XddUKrEOJIoom0/sxdJDNC35MNRSpV3dat7cy4ljVveXc24XsE83K8F3h7urg5LFUOBbRTlkbZRKOVaSclpvPTdDuZvOEyretX5960RhDbSwXllWXHbKBCRQUA7wCdrnzHmxZIJTylV0fhV8eSN4RH8Kaw+T3/1O8eSUjRRlGOOLFw0HagK9AE+BIYB65wcl1KqAujbuh5LHw+gipdV9fTl+kO0bVhDk0Y540i3hG7GmLuAs8aYF4CuQEvnhqWUqiiykkRKWgZTft3LTdNW8eYve7icnuniyJSjHEkUKbafl0SkIZAGNHBeSEqpisjH053vH+rBTRENmbLYShjbj+rg2PLAkUSxUET8sdbN3gjsB+Y4MSalVAXlV9WTN2+L5IO7OpJwIZVb3lmtq+mVA/m2UYiIG7DYGJMI/FdEvgN8clkaVSmlHNa/bT06NqvJ6rjT1KnuDcCJcynUq+FTwDOVK+RbojDGZALTsm2nluUkoXM9KVV+1PT1YlC4VYu96o8Eery2hCmL95KWoW0XZY0jVU+LRWSoiJT5CVyMMQuNMQ/4+WmPCqXKk7YNajAwtD5v/rKHm99Zxa7j51wdksrGkUQxFmvZ01QROSci50VEf4tKqRJT09eLKXe0Z/qoDhxPSmHw2yv5cEWZnKS6UipwHIUxpnppBKKUUgND69M5qBbPfrMNb0+d9qOscHRkdk2gBTlHZi93VlBKqcqrlq8XU0dEkTW90NebDnM0MYWxPYPxcNcZaV2hwHddRO4HlgM/AS/Yfj7v3LCUUpVdVrPo2vgzvP7Tbnr83xLmxR7KcczquASmL4tzRXiViiPp+RGgE3DAGNMHaA8kOjMopZTK8urQcKaOaM+F1HQmzd/Kk//dSnpGpn3VvfDG2nnF2RypekoxxqSICCLibYzZJSKtnB6ZUkrZ3BjekC7BtRn/2Qbmrj/E5fRMlu05xdQR7XOsuqecw5ESxWHbyOwFwC8i8g1wwJlBKaXU1QKqeTN3bFeGd2zM15uOMCq6KQ38qrg6rErBkV5PN9vuPi8iSwA/4EenRqWUUrlYE3+axTtP8nDf5sxac4BpS//glvaNeX5IO3y9Heqbo4rAkcbsplk3YB+wGajv7MCUUiq7rDaJqSPa85cBrZg6oj2e7m7M23CYQVNWsOVQoqtDrLAcqXr6HvjO9nMxEA/84MyglFLqalsPJ+Vok+jRog4fj+7EiM5NSU3PZOi7q3lPe0A5hSNVT2HZt0UkChjvtIiUUioX43qFXLOvW0gA3UICSLqUxtNfb+VCaroLIqv4Cl2pZ4zZKCLRzghGKaWKwq+qJ9NGRJFpjdFjdVwC51PSub6d1pKXBEeWQv1Ltk03IAo46rSIlFKqCEQEd9vUpR+v3MeinScZEd2Ufwxqa19lTxWNI20U1bPdvLHaKm5yZlBKKVUc74zswNiewcxZe5DBU1fqSnrFJFnzqVQEIjIYGNy8efMxe/fudXU4SikXW7k3gb98uZnES2l8PaEb7RrqKO68iMgGY0zHXB8rKFGIyELg6oOSgFjgPWNMyrXPcq2OHTua2NhYV4ehlCoDzly8zOfrDjK+dwgiQkamwd2tzC+vU+rySxSOVD3FAxeAD2y3c8B5oKVtWymlyqxavl5M6NMcEeHw2Uv0f3MZS3afdHVY5YojvZ66GWM6ZdteKCLrjTGdRGS7swJTSqmSdjk9Ey8PN+6ZsZ57YgJ5cmBrfHTdiwI5UqKoZhuVDVgjtYFqts3LTolKKaWcILhONRZMiGF0t0BmrNrPn6etYs+J864Oq8xzJFH8FVgpIktEZCmwAnhcRHyBWc4MTimlSpqPpzvPD2nHjNGdOHU+lRmr9rk6pDLPoV5PIuINtLZt7i6LDdjZaWO2UsoRJ8+nUNXLg2reHuxLuIhfFU9q+Xq5OiyXKG5jNsaYVGPMFtutTCcJpZRyVN3qPlTz9sAYw0Ofb6THa7/y/vKc80XpKnoOJgqllKrIRIRXbwnHr4on//zfLibO2cjldF1FL4tO4K6UUkBoIz8W/7U3E+ds5Lutx4jdf4aU9EzeGRlV6VfRc2Q9ChGRUSLyrG27qYh0dn5oSilVuqp4ufPR6E7cEFqf4+dSGdG5aaVPEuBY1dM7QFfgDtv2eWCa0yJSSikXWh2XwNp9Z3iob3O+WH+IX3ac4MMV8WRmVpzpjgrLkaqnaGNMlIhsAjDGnBWRytktQClVoWVfRa9bSABdQ2pz/6xYLl3OYMXeBN66LZKalbBXlCMlijQRccc235OI1AEynRqVUkq5wNWr6HULCeDDuzpyfbt6rIk7zY1vr2RzJVxy1ZFJAUcCt2GtQzELGAY8Y4yZ5/zwikbHUSilStrWw4k8+NlGTp5PYeqIqAq3KFJ+4ygcWQp1tohsAPoBAvzZGLOzhGNUSqkyLbyxP98/3J0Xv9tB+yb+rg6nVDnS6ykE2GeMmQZsA/qLiL+zA1NKqbLGv6oXb94aSd0aPmRkGp5Z8Dtxpy64Oiync6SN4r9Ahog0B94DmgBznBqVUkqVcQdOX+R/vx9nyNsr+X7rMVeH41SOJIpMY0w6cAsw1RgzCWjg3LBK2cq3YN/ynPv2Lbf2K6VULoLrVOP7h7vTqn51JszZyIsLd5CWUTH7+Tja6+kO4C7gO9s+T+eF5AKNomDe6CvJYt9ya7tRlCujUkqVcQ38qvDFA125JyaQj1ft47G5m10dklM4Mo7iHmAc8IoxZp+IBAGfOjesUhbUE4bPhE/+DG4ekHEZfOvA94+Dpw94VAEPb/CsAh4+V356+Fx5vKjHuZXjRVNWvmUl06CeV/btWw5HNkL3R10VlVKlysvDjecGt6NDs5o08q/i6nCcwpFeTzuAh7Nt7wNec2ZQLhHUEwK7w75lUD8c6raB9BRIS4H0ZEhLhuQzkJ6abZ/tZ2Z60a/r5plLQilsMvLO+XjWObKfJ/tx7iU0xVdWSWz4TOv9yyqJDZ9ZMudXqhy5Mbyh/f4r3+/Ar4on43s3x60CrM9d4CeGiLQA/gW0BXyy9htjgp0YV5GIyGBgcPPmzQv/5H3L4cQ26PkExH4E17+S85tyfjLSraSSnmIlFPv9qxJKeuqVxx09LvnsVcfZnpuZVvjXmMXNI5eE4mCSufq4TmPgixHQejDs/h8MeRuadi16bEqVc5mZhlPnU/lgxT42HkzkzVsj8K9avkdzOzLgbiXwHDAZGIxVFeVmjHnW+eEVTaEH3GX/Jnz1N2NHk0Vpy8wouWRU4HG2+xmFWPnWuwZU8YcqtaBKTahq+1mlVs772R/z8SvfVXFK2Rhj+Oy3A7z43Q7q1fDhnZFRhDf2d3VY+cpvwJ0jiWKDMaaDiPxujAnLvs8JsZaIQicKrWt3TFZyyp5k9q+EX/4BLQbA7h+g/SgrASSfsUpDl2w/s7aTE7HNBpMLsSWXq5PI1cnmqsTjXR2k/BfvVcWz6eBZJszeyIXUdFY91ZfqPmW3H1CxRmYDqSLiBuwVkYnAEaBaSQbocrklg6CeZbc04Spu7uDla93ASqaLX4DbPnO8JJaZASlJuSeRS2dy3r94ChJ2W8kl9Vw+cXlcSSA5Siw180k2tcCragm/QUrl1L5pTb57uAfbjiTZk8Tl9Ey8PMrXmnGOlCg6ATsBf+AloAbwujHmN6dHV0Q611MpKc2SWEaaLak4kGCSE69sp13K+5wePleVXnJLNrkkGI9i1jdrCbbSmhd7iA9X7OPdUVEE1ylb37eLVfVUHmmiUHZpyQ4kmMScj106k39nAa9qjpVYst/38bvS26w8tompErF8zyke+WITaRmG14eFc0NY2Rm7XNw2il+A4caYRNt2TeALY8z1JR1oSdFEoYrFGLh8Me92lktXJ5ts2yafkbk+flcSB8CJ7VbJ4vjvcP0/Ifw2qzeZqtCOJCYzYfZGNh9K5P7uQTx5Q2s83V1fFVXcRLHJGNO+oH1liSYK5RKZmVZbSo6Ekk8V2ek4SE268nxxg5qBENAK6rSCOq2hTktr27tsVVOo4rmcnskr3+9g1poDfHZfNN1buH651eI2ZmeKSFNjzEHbyZqRd7cVpSovNzdbry3/go/Nqm7q/heI/Rg6P2AlilO7IGEP/LEoZ/WXXxMIaGlLHq2u3KrUdNKLUc7k5eHGCzeFcmunJrRr6AfAyfMp1K1eNkuUjiSKvwMrRWQZ1noUPYAHnBqVUhXZ1W0SIX2ubPd52jomIx3O7oNTu63kcWq31QMs9mNrTEuWavVySSCtrSlotMtwmZeVJLYeTmT49DU83K8FD/YKKXOjuR1qzBaRAKCLbfM3Y0yCU6MqJq16UmVacXo9ZWZC0kFbAtl9JZEk7MnZhbhKzWursOq0hhqNNIGUQRdT03nqq99ZuOUo/VrX5c1bI/GrWrpjLkqs15OIPG+Meb6kAnMWTRSq0jEGzh+zlT72XCmFnNpltYtk8aqWexWWfzMdFe9ixhg+WXOAl7/fQX0/H94d2YHQRn6ldv2STBQbjTFlfu5tTRRKZXMxwZY4rkoiF45fOcbDB2q3uLYEUisY3D117Ecp2mgbzX1rxyY81r9lqV23JBNFme7tlEUThVIOSE60qqzspQ/bLenglWPcPKBWCFStDcc2QZeJEDYULpyE+ffo2A8nSbx0meo+nri7CbuPn6dprapU8XJuia+43WNjjDGrbPfdjDGZ2feVRZoolCqG1Atwem+2hnRbMjkTz5UOjwJBvSD8VqsxvkbD/M6oiij5cgY9X19CbV8vpo/qQGCAr9OuVdxEcU11U1mvgtJEoZQTpKXAj0/BhhlQp401F9clW7+WgFYQ3NtKGs1iwKeGS0OtSJbsPsljczeTkWF4fXgEA0PrO+U6RRpHISJdgW5AHRH5S7aHagDa6qVUZXN4Hez89sqaLUNngG8tiF8KcUtg4yew7j0Qd2jc6UriaNTBaudQRdKnVV2+e6g742dvZNxnG3igZzBPDmyNeyl2oc1vHIUX1iyxHkD1bPvPAcOcGZRSqoy5euxHUI8r290esm5pKVYyiVtiJY9lr8GyV8GrurV6ZEgfCO4DAS20i24hNa5ZlXnjuvLSdzvYn3CR0h5m4UjVUzNjzAHbfTegmjEmnzmfXU+rnpQqYUXp9XTpjHVM/FKIXwJn91v7azSyShtZt2p1nRh4xZOWkYmnuxuHzlziSGIyXYJrl8h5i9tGMQcYB2QA67Gqnv5jjHm9RKJzAk0USpVBZ/ZdSRrxyyAl0dpft92V0kazrlfWO1H5eujzTXy/9SiTrm/NuF7BSDFLacVNFJuNMZEiMhKIAp4CNhhjwosVlRNpolCqjMvMgGNbriSOg79ZS+26e0GTaAjuBcF9oWHklYGAOpYjhwup6Tw5fyvf/36MDs1q8vHoTvhVsdqCVsclsPVwEuN6hTh8vuImiu1AJDAHmGqMWSYiW4wxEQ5HUMo0UShVzly+BAfX2EobS62p18Gamj2op1Xa8PKFn/6m63hkY4zh2W+28+lvB6hXw5u5D3TlaFIyE+dsYuqI9nQLcXxW2uLOHjsd2A9sAZbbZo8t020USqlyxqsqNO9n3QAunIJ9y6zEEbcUdi609vvWhdnDoM1NELe4UicJABHhpT+HElLXl5e/38nc9QeZG3u40EmiIPkmClvj9QljTKNs+w4CfUosAqWUulq1OhA2zLoZY63dkVXa2PMz/P4leNeAfSusKdhrBbk6Ypca3S2IMxcuM+XXP3i4b/MSTRIA+S6rZIzJBJ64ap8xxqSXaBRKKZUXEQhoDp3HQPRY8KkOrQdby9wu/z+YEgkz/gSbZlujyiuh1XEJfLb2IA/3bc5naw+yOq5kJ/h2ZP29RSLyuIg0EZFaWbcSjUIppQqSvU3i9s/gzq+s6dTb32nNnPvNeHijJSwYD/tXWSWRSmB1XIK9TeIvA1oxdUR7Js7ZVKLJwpHG7H257DbGmOASi6KEaWO2UhVQfr2eYh6BQ2th02ew/Wu4fMFaVjZyJETcAf5NXBW1001fFkd4Y78c1U2l3uupPNJEoVQldvki7PgWNs+G/SsAsbrbRo6ENoPBs4qrIyyTits91hN4EMhK40uB94wxaXk+ycU0USilAGs0+ObPYfMca/p07xoQegtEjoLGHXUqkWyKmyg+BDyBWbZddwIZxpj7SzTKEqSJQimVQ2YmHFhpNXjv+MZadzygJUSOgPDboUYDV0focsVNFNcMriuPA+7S0tI4fPgwKSkpLopKKXU1Hx8fGjdujKdnKc4um3IOdiywksah30DcIKQftB9pdcNt0rlSjv4u7oC7DBEJMcbE2U4WjDXvU7ly+PBhqlevTmBgYLHnRFFKFZ8xhtOnT3P48GGCgkpxHIRPDYi6y7ol/AFb5ljVU/NGg6cvmAwY+C/ocI/VxpHV06oSc6RE0Q+YAcQDAjQD7jHGLHF+eEWTW4li586dtG7dWpOEUmWIMYZdu3bRpk0b1waSmWEN6Ns8x2oIz0wD/6ZW6eO2TyvF6O+iLlz0KLAaWAa0AFrZHtptjEkt6SDziSMY+DvgZ4wp1joYmiSUKlvKzP+kmzs0v866JZ+FefdYiQNgy1yo3aJSt2PkN+CuMfAWcBL4GbgdaAo4PAewiHwsIidFZNtV+weKyG4R+UNEnsrvHMaYeGPMfY5eUymliuX473B8K3R7GDx8YOsX8HYULPmX1fW2EsozURhjHjfGdAPqA08DZ4B7gG0issPB888EBmbfISLuwDTgBqAtcIeItBWRMBH57qpbhVnR5PTp00RGRhIZGUn9+vVp1KgRkZGR+Pv707Zt21KNZcGCBezYceVX+Oyzz7Jo0aJCn2f//v2EhoaWZGglplq1aqV6vZkzZzJx4sRr9qempnLdddcRGRnJ3LlzSzWmghTl95eYmMg777xj3z569CjDhlWgBS+zj/4e8BKMnGet0NewvbVa35Qo2PipVVVViTgyhUcVrMWK/Gy3o8BaR05ujFmOlWCy6wz8YSspXAa+AG4yxvxujLnxqttJh19JCZq+LO6a4e+r4xKYviyuyOesXbs2mzdvZvPmzYwbN47HHnvMvu3m5sivoXDS0/OejuvqRPHiiy9y3XXXlXgMpSW/1+pqmzZtAmDz5s3cdtttDj0nI8O1H0L5vZ9XJ4qGDRsyf/780girdBzZmHNG2qCecNsn0GIA3PuzNcL724nwXk+I+9WloZamPD+hROR9EVkFzAW6YrVXDDfGdDTG3FOMazYCDmXbPmzbl1cctUVkOtBeRJ7O57gHRCRWRGJPnTpVjPAgvLFfjrlSsuZSCW/sV6zz5iUjI4MxY8bQrl07BgwYQHJyMgBxcXEMHDiQDh060KNHD3bt2gVY3wT79u1LeHg4/fr14+DBgwCMHj2acePGER0dzRNPPJHr81evXs23337LpEmTiIyMJC4ujtGjR9v/2devX0+3bt2IiIigc+fOnD9/nv3799OjRw+ioqKIiopi9erV+b6epUuXcuONN9q3J06cyMyZMwEIDAzkiSeeICwsjM6dO/PHH3/kiL1jx460bNmS7777zv7eTJo0iU6dOhEeHs57771nv0aPHj0YMmRIniWyxx57jHbt2tGvXz+y/iY2b95Mly5dCA8P5+abb+bs2bMA9O7dm6wOEAkJCQQGBgJWSeGWW25h4MCBtGjRgieeuDJH5owZM2jZsiWdO3dm1apV11z/5MmTjBo1ivXr19vf68WLF9O+fXvCwsK49957SU1Ntb8vTz75JFFRUcybNy/Hefbt20fXrl0JCwvjmWeesZeW8nufX3zxRTp16kRoaCgPPPAAWZ1WNmzYQEREBBEREUybNs3+3JkzZzJkyBD69u1Lv379uHDhAv369SMqKoqwsDC++eYbAJ566ini4uKIjIxk0qRJOUolGRkZPP7444SGhhIeHs7bb79tf07btm0JDw/n8ccfz/V3VWZ0f/Tahuugntb+ptFw3y8wbAaknodPb4bPhsHJna6ItHQZY3K9AT8CsVjVRw8AYdh6SRXmBgQC27JtDwM+zLZ9J9aCSIU6b363Dh06mKvt2LEjx/at01dfc/tk9T5jjDGXUtPN9ZOXmZCnvzfd/rXIhDz9vbl+8jLz5fqDxhhjTl9Ivea5hfHcc8+Z119/3RhjzL59+4y7u7vZtGmTMcaY4cOHm08//dQYY0zfvn3Nnj17jDHG/Pbbb6ZPnz7GGGNuvPFGM3PmTGOMMR999JG56aabjDHG3H333WbQoEEmPT093+fffffdZt68efZ4srZTU1NNUFCQWbdunTHGmKSkJJOWlmYuXrxokpOTjTHG7Nmzx2S9v/v27TPt2rW75vUtWbLEDBo0yL49YcIEM2PGDGOMMc2aNTMvv/yyMcaYWbNm2Y+7++67zfXXX28yMjLMnj17TKNGjUxycrJ57733zEsvvWSMMSYlJcV06NDBxMfHmyVLlpiqVaua+Pj4XN9jwHz22WfGGGNeeOEFM2HCBGOMMWFhYWbp0qXGGGP+8Y9/mEceecQYY0yvXr3M+vXrjTHGnDp1yjRr1swYY8yMGTNMUFCQSUxMNMnJyaZp06bm4MGD5ujRo6ZJkybm5MmTJjU11XTr1s1+jbzei+TkZNO4cWOze/duY4wxd955p5k8ebL9fXnttddyfS2DBw82s2bNMsYYM3XqVOPr61vg+3z69Gn7/lGjRplvv/3W/vqXLVtmjDHm8ccft//+ZsyYYRo1amR/XlpamklKSrK/HyEhISYzM/Oa33n27XfeeccMHTrUpKWl2WNISEgwLVu2NJmZmcYYY86ePXvN67v6f7NcSEsxZuV/jPlnE2Oe9zfm20eMOX/C1VEVCxBr8vhMza+NYiDQCXjDtuuvwHoR+VlEXihGbjoCZJ+hq7FtX5niV8WTejW8OZKYQr0a3vYlBp0hKCiIyMhIADp06MD+/fu5cOECq1evZvjw4URGRjJ27FiOHTsGwJo1axgxYgQAd955JytXrrSfa/jw4bi7u+f7/Lzs3r2bBg0a0KlTJwBq1KiBh4cHaWlpjBkzhrCwMIYPH56j2qoo7rjjDvvPNWvW2PffeuutuLm50aJFC4KDg9m1axc///wzn3zyCZGRkURHR3P69Gn27t0LQOfOnfPsf+/m5mav6hk1ahQrV64kKSmJxMREevXqBcDdd9/N8uXLC4y3X79++Pn54ePjQ9u2bTlw4ABr166ld+/e1KlTBy8vL4eqlXbv3k1QUBAtW7bM9fp5nWPVqlX29+zOO+8s8DoAS5YsITo6mrCwMH799Ve2b99OYmIiiYmJ9OzZM9dz9e/fn1q1rImhjTH87W9/Izw8nOuuu44jR45w4sSJfK+5aNEixo4di4eH1ZmyVq1a9vftvvvu46uvvqJq1aoOxV/meXhDzMPw8CboNAY2fQpT2sPyN6zpzyuYfAfc2bLMNhFJBJJstxux2hmeK+I11wMtRCQIK0HcDowo4rmKbO7Yrnk+VsXLnUeua8HEOZvs87s/cl0L++yMtXy98n1+YXl7e9vvu7u7k5ycTGZmJv7+/mzevLlQ5/L1tTqlFfX5uZk8eTL16tVjy5YtZGZm4uPjk+/xHh4eZGZm2revHg2fvUtkXvezto0xvP3221x//fU5Hlu6dKn9tTqioG6Y2WO+Ot6rfz/OahPJ7/XkFn9e73NKSgrjx48nNjaWJk2a8Pzzzzs0I0H268+ePZtTp06xYcMGPD09CQwMLNKsBh4eHqxbt47Fixczf/58pk6dyq+/VqC6fd/a8Kf/g84PwC/Pwq8vQewM6PcshA0HJ7Q/ukJ+bRQPi8gXthXtlmEliF3ALYBD61GIyOfAGqCViBwWkfuMtejRROAnYCfwpTFmezFfR4kqjfndC1KjRg2CgoLs9dXGGLZs2QJAt27d+OKLLwDrH7pHjx6Fen716tU5f/78Nc9p1aoVx44dY/369QCcP3+e9PR0kpKSaNCgAW5ubnz66acFNrY2a9aMHTt2kJqaSmJiIosXL87xeFbvn7lz59K165WEO2/ePDIzM4mLiyM+Pp5WrVpx/fXX8+6775KWZs1BuWfPHi5eLLiLYmZmpr3dZc6cOXTv3h0/Pz9q1qzJihUrAPj000/tpYvAwEA2bNgA4FDjbHR0NMuWLeP06dOkpaVd066Qm1atWrF//357u0z26+cnJiYmx+87S17vc9YHekBAABcuXLC/Hn9/f/z9/e0l0OznulpSUhJ169bF09OTJUuWcODAASDvvx2wSiTvvfeePZGeOXOGCxcukJSUxJ/+9CcmT55s/xuscAKawx1zYPT34BsAXz8AH/SB/SsLfm45kF+JIhCYBzxmjMm/ziIPxpg78tj/P+B/RTlnadh6OCnHmrPdQgKYOqI9Ww8nlfgSg/mZPXs2Dz74IC+//DJpaWncfvvtRERE8Pbbb3PPPffw+uuvU6dOHWbMmFGo599+++2MGTOGKVOm5PhQ9PLyYu7cuTz00EMkJydTpUoVFi1axPjx4xk6dCiffPIJAwcOLPCbfJMmTbj11lsJDQ0lKCiI9u3b53j87NmzhIeH4+3tzeeff27f37RpUzp37sy5c+eYPn06Pj4+3H///ezfv5+oqCiMMdSpU4cFCxYU+N75+vqybt06Xn75ZerWrWtPTrNmzWLcuHFcunSJ4OBg+3v3+OOPc+utt/L+++8zaNCgAs/foEEDnn/+ebp27Yq/v7+96jA/Pj4+zJgxg+HDh5Oenk6nTp0YN25cgc/7z3/+w4gRI3jttde46aab7Pvzep/9/f0ZM2YMoaGh1K9f316VCFYD/L333ouIMGDAgDyvOXLkSAYPHkxYWBgdO3akdevWgNV7LyYmhtDQUG644QYmTJhgf87999/Pnj17CA8Px9PTkzFjxjB06FBuuukmUlJSMMbw5ptvFvh6y7XA7jBmCfw+Dxa/ADMHQatB0P9FK5mUUxVqPQoRGQwMbt68+ZiseuwsO3fudP00AYrAwEBiY2MJCMiZcEePHs2NN95YsfrkO0m1atW4cKHiLPlZYf8305JhzTRYORnSU6DjfdDrSau6qgzKbwqPilGBZmOMWWiMecDPzzndWJVSymGeVaDn41aDd/s7Yf0HVoP3qimQXmqzIJWIClWiyJLXpIAV8luLUuVcpfnfPLkTfv4H/PGLNeHgdS9Au5vLzOJJlaZEoZRSZVbdNjBqPtz5tbXS3vx74KP+cGidqyMrkCYKpZQqTSF9YexyGDIVEg9ZyeLLu+HMPldHlidNFEopVdrc3CHqTnhoA/R6Cvb+DNM6w09/t6Y5L2MqVKIQkcEi8n5SUpKrQ1FKqYJ5V4M+T8NDGyHsVquX1JT28Nt0SL/s6ujsKlSiKOu9ntzd3YmMjCQ0NJThw4dz6dKlHPuzbq+++ipgTVTXqlUrIiIiiImJYffu3YA14A6syQHnzJnjktcyc+ZMjh496pJrZ3HFNOfPP/88b7zxRsEHKlUYNRrAn6dZVVL1w+HHJ+GdLrBzIZSBDkcVKlGUmJVvWfPSZ7dvubW/GKpUqcLmzZvZtm0bXl5eTJ8+Pcf+rNtTT11Zy2n27Nls2bKFu+++m0mTJgHYZ291dqIwxuSYIiK7spAolKpwGoTDXd/AiHng5gFzR1mD9o5sdGlYmihy0yjKWrwkK1lkLWbSKKrELtGjRw/7VA6O6Nmzp/34rGmmn3rqKVasWEFkZCSTJ0/OcfyECRP49ttvAbj55pu59957Afj444/5+9//DsCbb75JaGgooaGhvPXWW4CVfFq1asVdd91FaGgohw4dYvTo0YSGhhIWFsbkyZOZP38+sbGxjBw5ksjISPu06Nm99tprhIWFERERYU98H3zwAZ06dSIiIoKhQ4faS1Tz5s0jNDSUiIgI+4R1eU0vfrX09HRGjhxJmzZtGDZsmP2c+U3nnZBgTcUSGxtL7969AaukcO+999K7d2+Cg4OZMmWK/RqvvPIKLVu2pHv37vZSnVJOIwItB8CDq2HQm3BqtzUdyH/HWI3fLpDvpIAV1g9PWcsd5qd6A2u++eoN4PwxqNMalr5m3XJTPwxueNWhy6enp/PDDz8wcKC1+F9ycnKOKSCefvrpa2YSXbhwIWFhYTn2vfrqq7zxxhv2tRuy69GjBytWrGDIkCEcOXLEPnPsihUruP3229mwYQMzZsxg7dq1GGOIjo6mV69e1KxZk7179zJr1iy6dOnChg0bOHLkCNu2WavZJiYm4u/vz9SpU3njjTfo2PHabtc//PAD33zzDWvXrqVq1aqcOWOtXXXLLbcwZswYAJ555hk++ugjHnroIV588UV++uknGjVqRGJiIgAfffQRfn5+rF+/ntTUVGJiYhgwYMA1s8Xu3r2bjz76iJiYGO69917eeecdJk6cyOjRo1m8eDEtW7bkrrvu4t133+XRRx/N9/eya9culixZwvnz52nVqhUPPvggW7du5YsvvmDz5s2kp6cTFRVFhw4d8j2PUiXC3QM63WdNLrhyMvz2Duz8FrqMh+6PgU+NUgtFSxR58fG3kkTSIeunj3+xT5mVEDp27EjTpk257z5rKfCrq56yJ4msb+2rVq0qVN14VqLYsWMHbdu2pV69ehw7dow1a9bQrVs3Vq5cyc0334yvry/VqlXjlltusU+W16xZM7p06QJAcHAw8fHxPPTQQ/z444/UqFHwH+eiRYu455577FNKZ01dvW3bNnr06EFYWBizZ89m+3ZrLsiYmBhGjx7NBx98YJ9wML/pxbNr0qQJMTExwJXpxAuazjsvgwYNwtvbm4CAAOrWrcuJEydYsWIFN998M1WrVqVGjRoMGTKkwPMoVaJ8asB1z8HEWGh7E6x802rwXv8RZJTO6o6Vs0ThyDf/rOqmnk9A7EfQ+8lrV74qpKyEUBizZ8/O9Vt7QbK+nf/444/07NmTM2fO8OWXX1KtWjWqV6+e73OzT/pXs2ZNtmzZwk8//cT06dP58ssv+fjjj3Mcv3btWsaOHQtYK6vlZfTo0SxYsICIiAhmzpzJ0qVLAZg+fTpr167l+++/p0OHDmzYsCHP6cWvltvU5PkpC9OJK1Uk/k3glvchehz8/Ax8/xdY+x407gjht0FwtpmI9y232jW6P1oil65QJYoS6x6bfYH1vn+3fmZvsygj8pvyGaBLly689dZb9OzZkx49evDGG2/YpyTv0aMHCxYs4NKlS1y8eJGvv/461+nKExISyMzMZOjQobz88sts3LjxmmtHR0fbS0NDhgyhf//+zJgxw95ekFX1dP78eRo0aEBaWlqOKa7j4uKIjo7mxRdfpE6dOhw6dMjh6cUPHjxoX/woazrx/Kbzzj6d+H//+98C3+OePXuyYMECkpOTOX/+PAsXLizwOUo5VaMoazrz22ZDZjpsng2fDYUNs6zHndCmWqESRYl1j81tgfXhM53W8yCrSirrlr3XU37Cw8Nxd3cnIiLimsZssJJBeno6zZs3JyoqijNnztiTQVRUFKNHj6Zz585ER0dz//33XzMdOMCRI0fo3bs3kZGRjBo1in/961/AlTWuc2vMHjhwIEOGDKFjx45ERkbaq8xeeukloqOjiYmJsU9bDTBp0iTCwsIIDQ21r9d9//3307ZtW6KioggNDWXs2LG5fsNv1aoV06ZNo02bNpw9e5YHH3wwx3TeYWFhuLm52afzfu6553jkkUfo2LEj7u7uBb7HUVFR3HbbbURERHDDDTfkmLJbKZcRgTY3wvjfYOBr1op7Cx+Gd7tbo7yzf36VxOV0UkCllCvp/2YJSD4Ls4fD4fVWQ/d1zxf6FDopoFJKVWTHf4cz8dD9r7DxkxKvJtdEoZRS5Vn2NtXrnnVKm2qlShQVsZpNqfJM/ydLQCm0qVaa7rE+Pj6cPn2a2rVrF9iFUinlfMYYTp8+jY+Pj6tDKd9y6wIb1LNEG7MrTaJo3Lgxhw8f5tSpU64ORSll4+PjQ+PGjV0dhipAhUoUIjIYGNy8efNrHvP09Lxm+gellFIFq1BtFGV9mnGllCqPKlSiUEopVfI0USillMpXhRyZLSKngAO2TT+gMJM/BQAJJR5UxVfY97ksKAsxl0YMJX2Nkjpfcc5TlOfqZ0H+mhlj6uT2QIVMFNmJyPvGmAcKcXxsXsPYVd4K+z6XBWUh5tKIoaSvUVLnK855ivJc/SwouspQ9aTTfZaO8vg+l4WYSyOGkr5GSZ2vOOcpynPLwu+7XKrwJYrC0m8RSinQz4LsKkOJorDed3UASqkyQT8LbLREoZRSKl9aolBKKZUvTRRKKaXypYlCKaVUvjRR5ENEfEVkloh8ICIjXR2PUso1RCRYRD4SkfmujsUVKl2iEJGPReSkiGy7av9AEdktIn+IyFO23bcA840xY4AhpR6sUsppCvNZYIyJN8bc55pIXa/SJQpgJjAw+w4RcQemATcAbYE7RKQt0Bg4ZDssoxRjVEo530wc/yyo1CpdojDGLAfOXLW7M/CH7VvDZeAL4CbgMFaygEr4XilVkRXys6BS0w8/SyOulBzAShCNgK+AoSLyLjr8X6nKINfPAhGpLSLTgfYi8rRrQnOdCrXCXUkzxlwE7nF1HEop1zLGnAbGuToOV9ESheUI0CTbdmPbPqVU5aKfBbnQRGFZD7QQkSAR8QJuB751cUxKqdKnnwW5qHSJQkQ+B9YArUTksIjcZ4xJByYCPwE7gS+NMdtdGadSyrn0s8BxOimgUkqpfFW6EoVSSqnC0UShlFIqX5oolFJK5UsThVJKqXxpolBKKZUvTRRKKaXypYlClTgRueDAMY+KSNUSvOafs8/yKSIvish1JXV+ZxERfxEZXwLneVRE7rLdnykiw4ofXfGJyGgRmVrAMTeKyIulFZMqPE0UylUeBQqVKGxTQOflz1jTQgNgjHnWGLOoSJGVLn+gWIlCRDyAe4E5JRGQC3wPDC7JLw6qZGmiUE4jIr1FZKmIzBeRXSIyWywPAw2BJSKyxHbsABFZIyIbRWSeiFSz7d8vIq+JyEZguIiMEZH1IrJFRP4rIlVFpBvWwlKvi8hmEQnJ/q1aRPqJyCYR+d22WI13tnO/YLvm7yLSOpfX4C4ib4jINhHZKiIPOXDOANv9jiKy1Hb/edtxS0Uk3vYeALwKhNjifv2qaweKyE7bCovbReRnEamSy1vdF9hoG1V8dfx5xfkn2+9kg4hMEZHvcnluOxFZZ4ttq4i0sO2/y7a9RUQ+te0bLCJrbddaJCL1cjlfHdvvbL3tFgNgrFG/S4Ebc3ltqiwwxuhNbyV6Ay7YfvYGkrAmVnPDmi6hu+2x/UCA7X4AsBzwtW0/CTyb7bgnsp27drb7LwMP2e7PBIZle2wmMAzwwZo2uqVt/yfAo9nOnfX88cCHubyWB4H5gIdtu5YD58x6XR2Bpbb7zwOrAW/b6z0NeAKBwLY83sdAIB2ItG1/CYzK5bgXsl6HI6892/4g2/7Pge9yOe/bwEjbfS+gCtAO2JPtNday/azJlZke7gf+bbs/Gphquz8n2++/KbAz27VGAm+7+m9Xb7nftEShnG2dMeawMSYT2Iz14Xe1LljVRqtEZDNwN9As2+Nzs90PFZEVIvI71odLuwKu3wrYZ4zZY9ueBfTM9vhXtp8b8ojtOuA9Y/u2bow548A58/K9MSbVGJMAnASu+dadi33GmM0FxNgAOJXL/rzibA3EG2P22fZ/nse11wB/E5EngWbGmGSs0ss822vIej/A+jLwk+33Moncfy/XAVNtv+NvgRpZJUes96NhHnEoF9P1KJSzpWa7n0Huf3MC/GKMuSOPc1zMdn8m8GdjzBYRGY1VaimJ+PKKrbDSuVKl65PHtQpzvaufk1vVU3Iu1yo2Y8wcEVkLDAL+JyJj8zn8beBNY8y3ItIbqwR1NTegizEmJZfHfLBehyqDtEShXOU8UN12/zcgRkSaA4iIr4i0zON51YFjIuKJVaLI7XzZ7QYCs84N3AksK0ScvwBjbQ3GiEitAs65H+hguz/UgfPnFXdh7ASa57I/rzh3A8EiEmjbf1tuJxWRYKySxxTgGyAc+BWrrai27ZhatsP9uLJuw915xPkz8FC280dme6wlsC2P5ykX00ShXOV94EcRWWKMOYVVl/25iGzFqvK4pmHZ5h/AWmAVsCvb/i+ASbbG1JCsnbZvr/cA82zVIpnA9ELE+SFwENgqIluAEQWc8wXgPyISi1UCyJexVk5bZWssf72g4/PwA7lUfeUVp60KaTzW+78BK1kl5XLeW4FttqqiUOATY025/QqwzPZ+vGk79nnbdTYACXnE+TDQ0dYQvoOcK8b1wer9pMognWZcqQpARL7GavTf6+Dx1YwxF0REgGnAXmPMZKcGmXcs9YA5xph+rri+KpiWKJSqGJ7CatR21BhbSWE7VrXRe84IykFNgb+68PqqAFqiUEoplS8tUSillMqXJgqllFL50kShlFIqX5oolFJK5UsThVJKqXxpolBKKZWv/we3726S0ZZG2QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot theoretical and PEPit (numerical) worst-case performance bounds as functions of the iteration count\n", "\n", "plt.plot(n_list, theoretical_taus, '--x', label='Theoretical upper bound for quadratics')\n", "plt.plot(n_list, pepit_taus, '-x', label='PEPit worst-case bound')\n", "\n", "plt.semilogy()\n", "plt.semilogx()\n", "plt.legend()\n", "plt.xlabel('Interation count n (log scale)')\n", "plt.ylabel('Worst-case guarantee (log scale)')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
↩ Back to TOC
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.4 Example 4 : point-SAGA" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We consider the following finite sum minimization problem\n", "\\begin{equation}\n", "F_\\star \\triangleq \\min_x \\left\\{F(x) \\equiv \\frac{1}{n} \\sum_{i=1}^{n} f_i(x)\\right\\},\n", "\\end{equation}\n", "where $f_1, \\ldots, f_n$ are $L$-smooth and $\\mu$-strongly convex functions with proximal operators readily available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The package PEPit allows to compute a tight (one-step) worst-case guarantee using a Lyapunov function for **point-SAGA**. The Lyapunov (or energy) function at a point $x$ is given by:\n", "\\begin{equation}\n", "V(x) = \\frac{1}{L \\mu}\\frac{1}{n} \\sum_{i \\leqslant n} \\|\\nabla f_i(x) - \\nabla f_i(x_\\star)\\|^2_2 + \\|x - x_\\star\\|^2_2,\n", "\\end{equation}\n", "where $x_\\star$ denotes the minimizer of $F$. The code computes the smallest possible $\\tau(n, L, \\mu)$ such that the guarantee\n", "\\begin{equation}\n", "\\mathbb{E}_{j_t}\\Big[V\\big(x^{(t+1)}_{j_t}\\big)\\Big] \\leqslant \\tau(n, L, \\mu) V\\left(x^{(t)}\\right),\n", "\\end{equation}\n", "is valid (for all initialization and all functions satisfying the assumptions)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "At each iteration $t$, point-SAGA picks $j_t \\in \\{1, \\ldots, n\\}$ uniformly at random and performs the following updates\n", "\\begin{equation*}\n", " \\begin{aligned}\n", " z_{j_t}^{(t)} &= x^{(t)} + \\gamma \\left(g_{j_t}^{(t)} - \\frac{1}{n} \\sum_i g_i^{(t)}\\right),\\\\\n", " x^{(t+1)}_{j_t} &= \\mathrm{prox}_{\\gamma f_{j_t}} \\left(z_{j_t}^{(t)}\\right),\\\\\n", " g_{j_t}^{(t+1)} &= \\frac{1}{\\gamma}\\left(z_{j_t}^{(t)} - x^{(t+1)}_{j_t}\\right),\n", " \\end{aligned}\n", "\\end{equation*}\n", "where $\\gamma=\\frac{\\sqrt{(n - 1)^2 + 4n\\frac{L}{\\mu}}}{2Ln} - \\frac{\\left(1 - \\frac{1}{n}\\right)}{2L}$ is the step size." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Theoretical guarantee" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A worst-case **upper** bound is provided by\n", "\\begin{equation}\n", "\\mathbb{E}_{j_t}\\left[V\\left(x^{(t+1)}_{j_t}\\right)\\right] \\leqslant \\frac{1}{1 + \\mu\\gamma} V\\left(x^{(t)}\\right).\n", "\\end{equation}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot worst-case guarantees as functions of the condition number" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have seen that the proof is based on a Lyapunov function. We plot the convergence rate for this Lyapunov function as a function of the condition number $\\kappa = \\frac{L}{\\mu}$." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# Set the parameters\n", "L = 1 # smoothness parameter\n", "n = 5 # the number of functions\n", "\n", "# Set a list of strong convexity parameter to test, given that the condition number kappa = L / mu\n", "mus = np.logspace(-4, -1, 14) \n", "\n", "# Compute numerical and theoretical (analytical) worst-case guarantees for each iteration count\n", "pepit_taus = list()\n", "theoretical_taus = list()\n", "for mu in mus:\n", " pepit_tau, theoretical_tau = wc_point_saga(L=L,\n", " mu=mu,\n", " n=n,\n", " verbose=verbose,\n", " )\n", " pepit_taus.append(pepit_tau)\n", " theoretical_taus.append(theoretical_tau)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABNcUlEQVR4nO3deVxW1fb48c8CFHDCAZwHEHHECXHCUNM0S69dzSk1NSvL0qZb3fo2e+tWt35lZmaWQ2pmapPNmbNhKjjlrCgqaCoqiDLIsH9/nAdCZDgqD6Cs9+vFS54zrocjz+KcvffaYoxBKaWUysmluANQSilVMmmCUEoplStNEEoppXKlCUIppVSuNEEopZTKlSYIpZRSuXJaghCRWSJyUkR25LFeRGSKiBwQke0iEpRt3WgR2e/4Gu2sGJVSSuXNmXcQc4A++ay/DQhwfI0DPgQQkarAS0BHoAPwkohUcWKcSimlcuG0BGGMWQOcyWeTO4C5xvIHUFlEagG3AsuMMWeMMWeBZeSfaJRSSjmBWzGeuw5wNNvraMeyvJbny9vb2/j6+hZmfEopdcOLiIiINcb45LauOBPENRORcViPp6hfvz7h4eHFHJFSSl1fRORwXuuKsxdTDFAv2+u6jmV5Lb+MMWaGMSbYGBPs45NrAlRKKXWVijNBLAVGOXozdQLijTHHgV+A3iJSxdE43duxTCmlVBFy2iMmEfkc6A54i0g0Vs+kMgDGmOnAj8DtwAEgEbjHse6MiPwH2OQ41CRjTH6N3UoppZzAaQnCGHNXAesN8HAe62YBs641htTUVKKjo0lOTr7WQyllm4eHB3Xr1qVMmTLFHYpS1+S6bqQuSHR0NBUrVsTX1xcRKe5wVClgjOH06dNER0fj5+dX3OGoG9j01ZG0qutFiL931rKwyFi2R8fzYDf/QjnHDV1qIzk5mWrVqmlyUEVGRKhWrZretSqna1XXiwkLthAWGQtYyWHCgi20qutVaOe4oRMEoMlBFTn9P6eKQvuYebzX8RwPzIvgnV/3MmHBFub2SCbk+PxCO8cNnyCK0+nTp2nTpg1t2rShZs2a1KlThzZt2lC5cmWaN29epLF888037Nq1K+v1iy++yG+//XbFx4mKiiIwMLAwQys0FSpUKNLzzZkzhwkTJhTpOdX1ZfrqyKy/8DOFRcYyfXVkvvslXUznUOwFwg7E8mVENMmp6QAs2nSU299bS9tJvzLq51Sa//4ogRe3MWXFAZ5tepLA3x+FOkH5HvtK3NBtEFfCGc/zqlWrxtatWwF4+eWXqVChAk8++SRRUVH069evMMK+RFpaGm5uuV/Sb775hn79+mUlpkmTJhX6+YtSfu9VqZIi8zHQ1OFtCfH3Jiwyloc/28wL/ZoTFhnL8bhk/jqXzOB2daleyYOvt0Qz6btdnE1MveQ4bepXxt+nAu5lXKhX0YVe1S8SUKY6h07dxuwTb3GyYiAVdx5gR69pBPp1LbT49Q7CoSie52WXnp7O/fffT4sWLejduzdJSUkAREZG0qdPH9q1a0doaCh79uwBrL/ce/ToQatWrejZsydHjhwBYMyYMTz44IN07NiRp59+Otf9w8LCWLp0KU899RRt2rQhMjKSMWPGsGTJEgA2bdpESEgIrVu3pkOHDiQkJBAVFUVoaChBQUEEBQURFhaW7/tZtWrVJUlvwoQJzJkzBwBfX1+efvppWrZsSYcOHThw4MAlsQcHB9O4cWO+//77rJ/NU089Rfv27WnVqhUfffRR1jlCQ0Pp379/nndgjz/+OC1atKBnz56cOnUKgK1bt9KpUydatWrFgAEDOHv2LADdu3fPGn0fGxtLZqmWOXPmMHDgQPr06UNAQABPP/101vFnz55N48aN6dChA7///ntBl1mVQukZhqNnElm7/xTuG97n7hqHGT/fegw0bm4ETZO3svfLVxn+8Qb+tXgbb/2yl8hTFwCoV6UcfQNr8FL3qszpkcayHjFs6RqB39p/waw+3LG8Jx8duZ3H946g345HCT6xiLKSTv2EzZxvOYpRKzwuu2O5FqXqT7ChH62/bFm/VrW4u7MvbetVoXpFd0bN3EiNSu6cOJdCo+oViDlrfXCfuXCR8fMjLtn3iwc6X3Us+/fv5/PPP+fjjz9myJAhfPnll4wcOZJx48Yxffp0AgIC2LBhAw899BArVqxg4sSJjB49mtGjRzNr1iweeeQRvvnmG8DqrRUWFoarqys9e/bMdf/+/fvTr18/Bg0adEkcFy9eZOjQoXzxxRe0b9+ec+fO4enpSfXq1Vm2bBkeHh7s37+fu+6665pKmXh5efHnn38yd+5cHnvssaxkEBUVxcaNG4mMjOTmm2/mwIEDzJ07Fy8vLzZt2kRKSgpdunShd+/eAGzevJkdO3bk2kPowoULBAcH8+677zJp0iReeeUVpk6dyqhRo3j//ffp1q0bL774Iq+88gqTJ0/ON96tW7eyZcsW3N3dadKkCRMnTsTNzY2XXnqJiIgIvLy8uPnmm2nbtu1V/0xU8bvaJwcZGYZj8UlExSZy6PQF2tStTMu6XuyIiWfgtDAupmcA0NmlPB+UeRmp8zKTV6TxUmAsww5/wOag/3KbT2VqZfxFlYvHKLt3FfwRRfDZwwTHHYa07J0cBCrVgSq+4N/D+reKL19HudGsQiJNw1+E4HupFz6TuT06sS46/pL3cy1KVYIoiJdnGWpUcicmLpk6lT3w8nReP3Y/Pz/atGkDQLt27YiKiuL8+fOEhYUxePDgrO1SUlIAWL9+PV999RUAd9999yV/1Q4ePBhXV9d898/L3r17qVWrFu3btwegUqVKgPVhO2HCBLZu3Yqrqyv79u27pvd71113Zf37+OOPZy0fMmQILi4uBAQE0LBhQ/bs2cOvv/7K9u3bs+5w4uPj2b9/P2XLlqVDhw55dh91cXFh6NChAIwcOZKBAwcSHx9PXFwc3bp1A2D06NGX/Hzy0rNnT7y8rLvH5s2bc/jwYWJjY+nevTuZZV2GDh16zT8XVbxyewSU+Tojw3D8XDJRsRfw8ixDYB0v4hIvMnj6eg6fSeRiWkbWcR67JYCWdb2oV6Uc99zki2+18vhVdqORS00ubDnLAzufZWSVulQ+cBjKlKPLhocuDcS9ElRpAD6NoXHvrCRAZV+oXA/c3C+LfUDFNbB4DAyeA35dwS+UwMVjCBw8Byicbq6lKkHk9xe/Z1lXHr0lgAkLtvBIj0bM33CER28JyMrEVcuXvaY7hpzc3f++4K6uriQlJZGRkUHlypWz2i3sKl++PMBV75+bd999lxo1arBt2zYyMjLw8PDId3s3NzcyMv7+hcnZzTN7z568vs98bYzh/fff59Zbb71k3apVq7Leqx0F9SbKHnPOeHNen7S0NNvnVdePEH9v3hrUiofmb2ZU5wbM33CEgOrleXnpTg6fTiTFkQSGBtfjzUGtqORRhoAaFejRtDoNqpXH17scflXKUiP1GOz8Bq+Tu3k2YTdE7obTkWCsxmWD4Jl0kOTydfkhsRnBbdvSwL/Z34nAswpcae+3mM1/Jwew/h08x1peSO0Q2gbhkP0vhyd6N2Hq8LaXtEkUhUqVKuHn58fixYsBa9DVtm3bAAgJCWHhwoUAfPbZZ4SGhl7R/hUrViQhIeGyfZo0acLx48fZtMmqbJKQkEBaWhrx8fHUqlULFxcX5s2bR3p6er6xN2jQgF27dpGSkkJcXBzLly+/ZP0XX3yR9W/nzn8n2sWLF5ORkUFkZCQHDx6kSZMm3HrrrXz44YekploNdfv27ePChQsF/PSsBJl517FgwQJuuukmvLy8qFKlCmvXrgVg3rx5WXcTvr6+RERYjw0z98tPx44dWb16NadPnyY1NTXr56yuL6v3nWLyb/t4YF44H742kU/mfUpFTzemrDjAyI71CcrYwTjX7xgd4strAwJZcF9H/nVrY8hIx+XsQaYFHefZCt8z/OgrhPzSn1of+OPyYUdYPBpWvwl/7QDvxhD6BMuav86BTq8j5apC16fxMEk0uWU0P1UeBoEDrR5H5apeeXIAuOmxyxOBX1dreSEpVXcQ+dkeHZ91mwnWXxZTh7dleyE+z7Pjs88+Y/z48bz66qukpqYybNgwWrduzfvvv88999zDW2+9hY+PD7Nnz76i/YcNG8b999/PlClTLvkwLFu2LF988QUTJ04kKSkJT09PfvvtNx566CHuvPNO5s6dS58+fQr8y71evXoMGTKEwMBA/Pz8Lns2f/bsWVq1aoW7uzuff/551vL69evToUMHzp07x/Tp0/Hw8OC+++4jKiqKoKAgjDH4+Phktbfkp3z58mzcuJFXX32V6tWrZyWlTz/9lAcffJDExEQaNmyY9bN78sknGTJkCDNmzKBv374FHr9WrVq8/PLLdO7cmcqVK2c9IlQlT3JqOvtPnGfX8Xh2H0/gfEoabw9uDcBHqyNZf/A0vtXK06x6Gz7+6xUeSXyUAT36seePH/mgzHuU6fc2uB+Bk7th+244tRtO7b20baByffBpBgG9oHoz68u7MZTxzNqk1yHnPwZyJrFKIl3/goODTc5G1N27d9OsWbNiikhl8vX1JTw8HG/vSxPtmDFjcm04vxHo/73CU1BD8qmEFPb+lUCXRlbVhNd/2s0naw+RnmF9tpUr60pgbS8WjuuEi4sQE5dEZc8ylHd3Y/3+4yxf8A7Pus7H1dufjBO7SDEueJKtm2nF2lC9KVRvDj6Z/zYBdxvjbtZNtu4Ssv+lf2iN9RioEP/SvxYiEmGMCc5tnd5BKKVKtJwNyQs2HGbS97sIqF6BT9YeIva8oyPHsz2o5eVJUP0qjO/mQvPalWhWqxINqpbDxcXxCOfcMeoc2wTRmyA6nPbRm+lsUiAN+OtPXLzqEVfzZjaaunS7qZuVEDwrX33wuSUBv66F1kbgbJoglNNFRUXlujxznIRSuUm8mMaWI3FsPHSGWl4ePDg/gjGdfZn5+yHS0g3pGdCtsY8jEVSkSrmyANzaoia3tqgJFxPh+DbY+3dCIOGYdXDXslCrDW4d7oOy5WDTJ9D+PgifRa1Og6l1nXyAO5smCKVUiWCMQUQ4eOo8jy/axs6YeNIyDA+6fUfLyi1p0qwLU1Yc4KHu/jwRcAK34xF//4VuDJw56EgEjmRwYgdkOHqfVW4ADUKgbnvrq2ag1XU0s41gyNy//7LP3mZQymmCUEoVOWMM0WeT2BR1xvF1ln+0qs2jtwTgU9Edd1cXxnVtSHu/qnTAk7Jf3cvDey7ySI/b2fPHj5gtkyH0cVj1JsSEW0khyRohT9kK1nP/Lo9ayaBOMFTIY0riIugqej3TBKGUuiZ2RiNnZBhOX7iIT0V3jDHc8s7qrPISFT3cCG5QhYY+5R2vy7Dowc7ZjtWKpRdH8aHr/3CN/BJjdiIpGfDbS4BY7QRN+znuDoKt1y6u9oK/ztsInE0ThFLqmuRVkO7RWwL4cFUk4VFnCD98lnpVPfl+Yigiwj/b1KGSZxna+1alSc2KuLrkGAeQmgRRv0Pkcppu/4kQc8hqSD7xJ1LVnyP1+rONxvzjtr7g4Zx6aUoThNO5urrSsmVL0tLSaNasGZ9++inlypXLWp5p2LBhPPPMM3Tv3p3jx4/j4eFBhQoVmDVrFk2aNCEkJISwsDCioqIICwtj+PDhRf5e5syZQ+/evaldu3aRnztTZiXcHTt2FNk5s1fiVZcL8fdmyrC2PDgvgjEhvszfcIQmNSvy8lKrvHxDn/LcFliTTg2rZe0zsWfApQcxxhpncOA3iFwOh8OsMQeu7lT17QJVesCOL6HD/RA+i/ptelBf/8p3Ok0QmZzUX9nT0zOr9MWIESOYPn06TzzxxCXLc/rss88IDg5mxowZPPXUUyxdujSrmmpUVBQLFixwWoIwxmCMwcXl8kH2c+bMITAwsFgThCo5LqSksXb/KZbtOsnKvSc5l5zGlBUHeKRHI7o08uZs4kWCfaviXeHyOkKA1WZwcLUjKayAczHWcu/GEDwW/HtaDcsx4VbD8dB52pBcxLTURqY6QdZ/ukNrrNeZvRsKcfKN0NDQrFLXdnTt2jVr+8zJcJ555hnWrl1LmzZtePfddy/Z/uGHH2bp0qUADBgwgLFjxwIwa9YsnnvuOQDeeecdAgMDCQwMzKpoGhUVRZMmTRg1ahSBgYEcPXqUMWPGEBgYSMuWLXn33XdZsmQJ4eHhjBgxgjZt2mSVJ8/uzTffpGXLlrRu3ZpnnnkGgI8//pj27dvTunVr7rzzThITEwGrxEZgYCCtW7ema1frlzyvMt85paWlMWLECJo1a8agQYOyjrl8+XLatm1Ly5YtGTt2bFahQl9fX2JjrZIp4eHhdO/eHbDuDMaOHUv37t1p2LAhU6ZMyTrHa6+9RuPGjbnpppvYu3evretVGmQOrF259yRtJy3jwfmb+W33CZrVrEgFdzfGd2vI/A1HSDeGPoG1Lk0OGelW76JVb8InveB/Da3yFLuWWr9n/3gPHtsBEzZBn9ch4BarC2p+DcnKqUrPHcRPz8Bff+a/TcVaMG+A9W/Ccauxa9Wb1lduaraE296wdfq0tDR++ukn+vTpA0BSUtIlpRqeffbZrEqkmb777rtLHkMBvPHGG7z99ttZ5bKzCw0NZe3atfTv35+YmBiOHz8OwNq1axk2bBgRERHMnj2bDRs2YIyhY8eOdOvWjSpVqrB//34+/fRTOnXqREREBDExMVmPceLi4qhcuTJTp07l7bffJjj48kGXP/30E99++y0bNmygXLlynDlzBoCBAwdy//33A/D8888zc+ZMJk6cyKRJk/jll1+oU6cOcXFxAMycOTPXMt85q7fu3buXmTNn0qVLF8aOHcu0adOYMGECY8aMYfny5TRu3JhRo0bx4Ycf8thjj+V7Xfbs2cPKlStJSEigSZMmjB8/nu3bt7Nw4UK2bt1KWloaQUFBtGvXLt/j3KiMMeyIOcey3SdYvvsEwzrU5+5ODQis7cWozg24pXkNamyfzuvbyzFj1N2E+HsT2tiHOZ/No3abRHy7j7UeGR34DQ6ucvQ0EqjdFkL/BY1usXoZuebzUaQNycWm9CQIOzwqW8kh/ih41bNeX6PsiSA0NJR7770XIN9HTCNGjMDT0xNfX1/ef/992+cKDQ1l8uTJ7Nq1i+bNm3P27FmOHz/O+vXrmTJlCrNmzWLAgAFZdZUGDhyYlVAaNGhAp06dAGjYsCEHDx5k4sSJ9O3bN2suhvz89ttv3HPPPZQrVw6AqlWrArBjxw6ef/554uLiOH/+fFaF1i5dujBmzBiGDBnCwIEDAfIs850zQdSrV48uXboAVlnvKVOm0KtXL/z8/GjcuDFglfX+4IMPCkwQffv2xd3dHXd3d6pXr86JEydYu3YtAwYMyHov/fv3L/D932gyMgwvLt3Bb7tO8te5ZFwEgupXoVp5azCaT0V3nu9nTdq0dLs/H5R5ljIu7SCtIyF/LaCjvMH5XVVh8/+sA1aoAY1vg0Y9oeHNUL5aXqdWJUjpSRB2/tLPfKzU9WkInwnd/33Nf6XklwjyktkGcaUy/xr/+eef6dq1K2fOnGHRokVUqFCBihUr5rtv9mJ8VapUYdu2bfzyyy9Mnz6dRYsWMWvWrEu237BhAw888ACQ//SlY8aM4ZtvvqF169bMmTOHVatWATB9+nQ2bNjADz/8QLt27YiIiMizzHdOuZUIz4+W9S64K2rs+RRW7DnJifhkJvYMwMVFOBR7gbb1K9OzWQ1ubuJDtTzaEvr3GwhVjsL8O8FkQEYaruKKV61G4P+AlRRqBF5dxVJVrLQNIlP2qos9nrP+zd4mUULkVbY7U6dOnZg8eTJdu3YlNDSUt99+O6s0eGhoKN988w2JiYlcuHCBr7/+Otey4bGxsWRkZHDnnXfy6quvsnnz5svO3bFjR7Zu3crWrVvp378/vXr1Yvbs2VntAZmPmBISEqhVqxapqal89tlnWeeIjIykY8eOTJo0CR8fH44ePWq7zPeRI0dYv96aHTCzrHeTJk2IiorKarPJq6z3l19+WeDPuGvXrnzzzTckJSWRkJDAd999V+A+JV1uU+qOn7+ZQ7EXGDjtd9q/9htPL9nOl5ujs4rczb+3Ix+ObMegdnUvTw4ZGVY31O8ehbcDYOWrIK7WyOUWd8IzR2D0d9bjoZotNTlcp0rPHURBinhEZc42iD59+vDGGwXf5bRq1QpXV1dat27NmDFjLpmdDawk8Ouvv9KoUSMaNGjAmTNnspJAUFAQY8aMoUOHDgDcd999tG3b9rJaSTExMdxzzz1Zf3W//vrrwN9zSHt6erJ+/Xo8Pf8ua9ynTx+2bt1KcHAwZcuW5fbbb+e///0v//nPf+jYsSM+Pj507NgxK8E89dRT7N+/H2MMPXv2pHXr1rRq1cpWme8mTZrwwQcfMHbsWJo3b8748ePx8PBg9uzZDB48mLS0NNq3b8+DDz4IwEsvvcS9997LCy+8kNVAnZ+goCCGDh1K69atqV69etZse9ezEH9vpt7VlgfmRTCyY32+CI+mV/MafLHpKC3rePFozwBuaVaDFrUrZd2RXXZnZoxVvuLPxfDnl3AuGsqUg6Z9rQqn66dCyETr7vuYjkS+EWi5b6WcoCT93zt5LpkvN8ewOPwoB2OtO7JHejRiVIgvaemGml75zxbI2Sj4c4mVGE7tARc3qwtqy8HQ9HaIibi022nOORBUiablvpUqheKTUvnXoq2s3HuK9AxD05oVKV/WlbsdU2t28q+W92RYF2Jh59dWUji6wVpWrxP0/X/QfMCljcxaz+iGpQlCqRvIgZPnOXDyPH0Ca1LJw43zKWmM69qQgOoVePWH3Xw8OpgQf2+6Nva5pDwGACnnYe+PVlI4sNyaT9mnGfR8EQIHQZUGuZ9Uu6HesDRBKHWdO5+Sxg/bj7EoPJqIw2epVr4stzSrjpurCwvHWUXv/pj7AnN7dCAw25S6c3skc2Hdu5DeFbYvspJDaiJUqmu1JbQcbJXFVqXWDZ8gMmvMK1VUirJd78uIaF74dgeJF9Px9ynP/93elAFt6+LmemkHxU6hvax2gdpe0OAm2DiDwJUvgUsZWDAVPKtAq6HQaoj1KCmXUiuq9HFqghCRPsB7gCvwiTHmjRzrGwCzAB/gDDDSGBPtWPc/oC9WV9xlwKPmCn/zPDw8OH36NNWqVdMkoYqEMYbTp0/j4VFAw+9VOpmQzFebY7ipkTeBdbwIqFGBf7SqzZD2dQmqXyXv/+d+XaHvu7BgiJUUUs6BS1lo2htaDgH/HuBW1ikxq+uX0xKEiLgCHwC9gGhgk4gsNcbsyrbZ28BcY8ynItIDeB24W0RCgC5AK8d264BuwKoriaFu3bpER0dz6tSpa3szSl0BDw8P6tate8X75TWYbeuROAJqVOSLTUdZufck6RmGp/s0IbCOF63qVqbVoMr5H/jETlg/Df5cBOkXgSRo0hcGfgTu+Q+gVKWbM+8gOgAHjDEHAURkIXAHkD1BNAeecHy/EvjG8b0BPICygABlgBNXGkCZMmUuK9OgVEmV27wKExZswbOMCzFxyXhXcOe+UD+GBNfD36dC/gczxmpoXj8VDq60xis06gWHf4cO4xxjFbZoQ7LKlzMTRB3gaLbX0UDHHNtsAwZiPYYaAFQUkWrGmPUishI4jpUgphpjduc8gYiMA8YB1K9fv/DfgVJFKMTfm3eHtub+T8OpU8WT2PMXmTq8LUdOJ1Ktgjvdm/hQxrWAtoHUJKvB+Y9p1piFirWg50vg3QS+m5itZHaojlVQBSruRuongakiMgZYA8QA6SLSCGgGZN6nLxORUGPM2uw7G2NmADPAGihXZFErVciSLqbz+cYjzFhzkAsX09l34jz3dvElxN+bEH8bBzh/EjbNhE2fQGKsVd5iwAxoMcBqW1g3WccqqCvmzAQRA9TL9rquY1kWY8wxrDsIRKQCcKcxJk5E7gf+MMacd6z7CegMXJIglLoR7P0rgRGf/EHs+Ys0rVmRCxfTGBPiy2cbjtCzeY28B7MBnNwN6z+w7hrSU6yKqZ0fBt+bLq1/pGMV1FVwZoLYBASIiB9WYhgGXDINmoh4A2eMMRnAs1g9mgCOAPeLyOtYj5i6AZOdGKtSRSo+MZXI2PME1a9CQ5/ydG3sQ2BtL6auPMBHd7cjxN+bzv7VLh/MBlb7QuQKKzFELgc3T2g7EjqNB++AvE+q1BVyai0mEbkd64PdFZhljHlNRCYB4caYpSIyCKvnksF6xPSwMSbF0QNqGtDVse5nY8wTuZ7EIbdaTEqVNKfPpzBz3SHmrj9MeXdX1v27R1a7QkEluUlNtkY5r/8ATu225ljoMM6anrNc1eJ6S+o6l18tphu6WJ9SJcXJc8l8tOYgCzYcITktndtb1uLh7o1oXrtSwTtfiHW0L3wMF05BjZbWY6TAgeCWx3zPStmkxfqUKmZ7/kpgTlgUd7SuzUM3+9Ooei7jD9ZNtuZmzmwXOLkHlr3wd12kgFutxODXVedXUEVCE4RSThAVe4Fpqw5QrYI7/+7TlNAAb9Y8fTN1KnvmvVOdIKvraZfHrJLZB5ZZyxv3gV7/AZ/GRRG6Ulk0QShViPafSOCDlQdYuu0Ybq4ujO1iDdQUkfyTA4B7JahY27prKFPO+hr4MTTrVwSRK3U5TRBKFZJZ6w7xnx924VnGlftCG3JfqB/VK9qoyRQfDcv/A9sXQrlq4H8LRP5mzY2uyUEVI00QStmUWy+jueujiD6bxP/d3owujbx5uHsjxt7kR9XyNgrfJZ+Dde9ao56NgZseh3od4duHreQQPtMa8axjFVQx0QShlE3ZayW5iPDq97vYcewc3RtbCaNJzYo0qdmk4AOlp0LEHFj1hjXqudVQ6PECnD10afkLLYehipkmCKVsCvH35rnbmzJq1kbS0g0CDO9Yj+dub27vAMbA3p9g2Ytwer81L8Otr0Ltttb6HV9qOQxVomiCUOoK7DyWQGa5vPHd/Xm6T1N7O8Zshl9fgMProFoA3LXQ6p2k5TBUCabTRilVgGW7ThBx+CwAXRpVo5y7G4/0aMTCTUcJi4zNf+e4I/Dl/fDxzVZ11b7/Dx5aD01u07EMqsSzdQfhmPktwBjzm4h4Am7GmATnhqZU8TpyOpFXvtvJ8j0n6duyFilp6Ty1ZDvTRgQR4u9Np7xqJQEkx8Pad+CPD61EEPova3yDh42R00qVEAUmCEdl1XFAVcAfqyrrdKCnc0NTqngkp6bz0eqDTFt1ADcX4bnbmzGmiy8z1x26JBmE+HszdXhbtkfH/50g0lMhfDasfgMST0Pru6DH8+B15TPMKVXc7NxBPIw1O9wGAGPMfhGp7tSolCpGX22O4d3f9tG3VS1e6Nucml7WWIYHu10+MYM1X4O31QC95werAfpMJPiGQu9XoXabIo5eqcJjJ0GkGGMuZk6GLiJuWBVWlbphxMQlceR0Ip39qzEkuC7+PuXp2LCavZ2jI+DX5+FIGHg3hru+gMa3ahuDuu7ZSRCrReT/AE8R6QU8BHzn3LCUKhoX0zL4ZN1B3l9+AO+KZVn15M24ubrknhxyFtM7exi+nQBRa6C8D/R9B4JGg6t2DlQ3Bjv/k58B7gX+BB4AfgQ+cWZQShWFsAOxvPDtDiJPXaB38xq80K85ri75/NWfWUzvjg/gcJg1AjojDVoOsXonaQO0usEUmCCMMRkiMh9YY4zZWwQxKeV0247GMfyTDdSvWo5ZY4Lp0bRGwTv5dYUuj8LC4WAywNUdBn1szcug1A2owHEQItIf2Ar87HjdRkSWOjkupQpdWnoGW45Y4xla1fXinSGt+fXxrvaSw8UL8P0TViO0RxVrWZdHNTmoG5qdgXIvYfViigMwxmwF/JwXklKFb1PUGfq9v45hM/7gxLlkRISBQXXxKONa8M5H/oAPu0D4LGj+TxDzdzG9Q2ucHrtSxcVOgkg1xsTnWKa9mFSJNH115CWjm08lpHD3zA0Mnr6ehOQ0ptzVluoVbU7TmZpslceY1cd6pHTrfyFqLQz+FHo8Z9VJWjxGk4S6YdlJEDtFZDjgKiIBIvI+EObkuJS6KpkVV8MiYzmXnMrNb69k7f5Y7mhdm2VPdOXWFjURO91Pj22FGd0hbAq0Gw3jf4f0i3kX01PqBiTG5H8zICLlgOeA3o5FvwD/McakODm2KxIcHGzCw8OLOwxVAizbdYJ/f7mdkR3r88m6Q0zq34JBwfXs7ZyeapXIWPM/KOcNd0yFgF7ODVipYiQiEcaY4NzW2enm2tcY8xxWksg84GBgcSHFp1Sh+XZrDM9/vYNbmtdgyooDPNKjkf3kcGovfP0AHNsCLQfDbf+DclWdG7BSJZidR0zP2lymVLFJupjOv5ds59GFW6lV2YOVe07ySI9GzN9wpOCKqxkZEDYVpodag98Gfwp3fqLJQZV6ed5BiMhtwO1AHRGZkm1VJSDN2YEpZdf+Ewk8vGAz+06c547WtVmz/xTTRtqouApw5pA1xefh36HJ7fCP96CClhpTCvJ/xHQMCAf6AxHZlicAjzszKKWuxNr9sZw+f5FPx3Zg9/FzDO1QL/+Kq2AV14uYA788By6u8M8PrcqrWj9JqSx2GqnLGGNSiyieq6aN1KXLhZQ09p88T5t6lTHGcDYxlarly9rb+dxxWDoRDiwDv25W6YzKNtsplLrBXGsjdQcReRlo4NheAGOMaVh4ISpl356/zvHwZ5s5feEi6/7dgwrubvaSgzHw5xL48UlIS4Hb3oL294GLTqyoVG7sJIiZWI+UIoB054ajVN6MMSzcdJSXl+6kkmcZpg0PooK7zcqpF07DD4/Drm+hbnv453TwbuTcgJW6ztn57Yo3xvzk9EiUykdqegb/WrSNpduOERrgzTtD2uCT24jonCW5AVa/BevetQa69XzJqqHkYqPEhlKlnJ0EsVJE3gK+ArIGxxljdPioKjJlXF1wd3Phyd6Neah7I1zyKsudWZJ78Byo1RoW3wORy6GKHwydDzUDizJspa5rdhqpV+ay2BhjehR4cJE+wHuAK/CJMeaNHOsbALMAH+AMMNIYE+1YVx9r3ol6WLWfbjfGROV1Lm2kvvEYY5i/4Qgd/arSuEZFjDH2ymQcWgMLR1htDhcTrPka7vgA3Gw2YitVilxTI7Ux5uarPKkr8AHQC4gGNonIUmPMrmybvQ3MNcZ8KiI9gNeBux3r5gKvGWOWiUgFIONq4lDXp/ikVJ79ajs//vkXY0J8ebl/C3vJwRiIiYCUc9brtiOt5KCUumK2WvhEpC/QAvDIXGaMmVTAbh2AA8aYg45jLATuALIniObAE47vVwLfOLZtDrgZY5Y5znXeTpzqxrDtaBwTPt/M8bhknr2tKfeH2uwwl5oM3z0C278A17LQ6WHYMte6o8jeJqGUssXOhEHTgaHARKwuroOxurwWpA5wNNvraMey7LYBmTOuDAAqikg1oDEQJyJficgWEXnLcUeibnB/HDzNoOlhZGTAFw905oFu/nm3N2SX8BfM6WslhzLlYMQS6PWyluRW6hrY6QAeYowZBZw1xrwCdMb6AC8MTwLdRGQL0A2IwepK6waEOta3BxoCY3LuLCLjRCRcRMJPnTpVSCGp4hRUvwpjb/Ljh0duol2DKvZ2itkMM26Gk7uh1TAY/gU07Gat05LcSl01Owki2fFvoojUBlKBWjb2i8FqYM5U17EsizHmmDFmoDGmLY5qscaYOKy7ja3GmIPGmDSsR09BOU9gjJlhjAk2xgT7+PjYCEkVt5wT+gDMCTtE6JsriE9MpaybC8/e1ozK5Ww2KO/4EmbfZnVbvfcXGPjR5Y+T/LrCTY8VzhtQqhSxkyC+E5HKwFvAZiAKWGBjv01AgIj4iUhZYBhwyVzWIuItIpkxPIvVoylz38oikvmp34NL2y7UdSr7hD4ZGYZnv9rOy0t3kZyWzqnzyQUfIFNGBqx4FZaMhdpt4f6VULOl8wJXqhTKt5Ha8eG93PFX/Zci8j3gkcsUpJcxxqSJyASsCYZcgVnGmJ0iMgkIN8YsBboDr4uIAdYADzv2TReRJ4HlYnVdiQA+vto3qUqOzOJ5D322GS/PMhw+nUhHvyp8PLo9lTzK2DtIynlr3oY931u9lPq+A242pxFVStlmZxzEFscjoBJNx0FcX26dvIa9fyXQvYkPs8e0t9eFFSDuCHx+F5zcBb1fg07jtQKrUtcgv3EQdh4xLReRO8X2b7BS+QuLjOXEuWQGtavL9uh41h88bW/Hw+utxui4ozBiMXR+SJODUk5kZxzEA1hjFdJEJJm/q7lWcmpk6obzx8HTvPPrXvafPM+0EdaEPmGRsflP6JNp81z4/gmoXN/qpeQdUHSBK1VK2RlJXbEoAlE3th//PM5jC7dS0cON1wYEFjyhT6b0NPj1edjwITS8GQbPBk+b3V+VUtfE7kjqKkAAl46k1pFHypZ566N4celOgupXYebo4Mu6sIb4e+eeHJLiYMk9ELkCOo6H3q+Cq83y3kqpa1bgb5uI3Ac8ijWOYSvQCViP1fVUqXzNWBPJf3/cwy3NajB1eFs8ytgcEB97AD4fCmcPwz+mQLvRzg1UKXUZO43Uj2KNZj7sKNzXFohzZlDqxhEa4MPYLn5MHxlkPzkcWA6f9ICkszB6qSYHpYqJrZHUxphkABFxN8bsAZo4Nyx1PUu6mM6iTUcxxtCsViVe/Edz3Fxt/FczBv74ED4bBJXqWoPfGoQ4P2ClVK7sPNCNdoyk/gZYJiJngcPODEpdv85euMjYTzex7WgcLepUokVtL3s7pl2EH56ALfOgSV8YOAPcKzg3WKVUvuz0Yhrg+PZlx+RBXsDPTo1KXZeizyYyatZGos8mMW1EO/vJ4fwpWHQ3HFkPoU/Czc+Bi52bW6WUM9lppK6f7eUhx781gSNOiUhdl/b8dY7RszaSdDGd+fd2pINf1dw3zDln9F87YN4ASDoDd86EloOKLGalVP7sPGL6AWvKT8Hq5uoH7MWaQEgpAKJiL+AqwuIHQ2hSM5+hM9nnjE4+B0vuhYyLcPvbmhyUKmHsPGK6pESmiAQBDzktInVdOR6fRC0vT/oE1qJ7k+oF91TKnJ9hwTBIvQAubnDnLAgckP9+Sqkid8UPeo0xm4GOTohFXWfm/XGYbv9bxcZDZwDsd2M9tsVKDgAhEzU5KFVC2WmDeCLbSxesiXuOOS0iVeIZY3hn2T7eX3GAW5pVp2Udm43RAGvesuZxcC1rJYeIOeDfQ+eMVqoEstMGkf2BchpWm8SXzglHlXRp6Rk8/80OFm46ytDgerw2IND+GIdVr8PqN8HV3Sq4538zNOz+d5uEJgmlShQ7bRCvFEUg6vrw/fbjLNx0lIk9GvFEr8b25nEwBpZPgnXvQK22cMvL4N/dWpd9zmhNEEqVKHYeMX2H1Yspu3ggHPgoc5S1urEZYxAR7mhTm+qV3PMvzX3pjrDsBQh7H9qNgb7vXj7Gwa+rJgelSiA7jdQHgfNYU35+DJwDEoDG6DSgN6TpqyMJi4zNeh0Tl8Rt763lvz/sQkSuLDn8/IyVHDqMg36TdQCcUtcRO20QIcaY9tlefycim4wx7UVkp7MCU8WnVV2vrEl8qpV3Z9iM9cQlpjK0fT37B8nIgB+fhPCZ0OlhuPU1nf1NqeuMnQRRQUTqG2OOQNbI6swiORedFpkqNpmT+DwwL4KU1HRSMwxv3NmSoe3rF7wzWMnhu0esukpdHrPaHDQ5KHXdsZMg/gWsE5FIrNHUfsBDIlIe+NSZwaniU8mjDEkX00nLMIwJaXAFySEdvn0Ytn0OXZ+Gm/9Pk4NS1yk7vZh+FJEAoKlj0d5sDdOTnRWYKl4nE5JxdRFGh/jy9ZYYereoWXDbQ3oafP0A7FhiFdzr9nTRBKuUcgpb8zcaY1KAbU6ORZUAR04ncuBUAk8u3s7se9oT4u9Nz2bVs9ok8kwS6anw5b2w61vo+RKEPpH7dkqp64ZO8KuyxMQlMWzGejzLul6SDDLbJLZHx+eeINJSYPE9sPcH6P0ahEwo4siVUs6gCUIBEHs+hbs/2UBCchozRgUTmKN8Roi/d+7JITUZFo2C/b/AbW9Bx3FFFLFSytkK7JQulpEi8qLjdX0R6eD80FRRiU9KZdTMjRyLT2LWPe0vSw55Sk2ChcOt5NDvXU0OSt1g7IxamgZ0Bu5yvE4APnBaRKrI/d9Xf7L/ZAIf3R1Me988JvrJ6eIFWDAEIldA/6kQPNa5QSqlipydR0wdjTFBIrIFwBhzVkTKOjkuVYSeua0pA4Pq0K2xj70dUhJgwVBritABH0Hroc4NUClVLOzcQaSKiCuOekwi4gNkODUq5XRp6RksDj9KRoahXtVy9GxWw96Oyedg/p1w5A8Y+LEmB6VuYHbuIKYAXwPVReQ1YBDwvFOjUk6VkWF45qs/WRIRTY1KHnS1e+eQFAfzB8LxbTB4NjS/w6lxKqWKV4F3EMaYz4CngdeB48A/jTGL7RxcRPqIyF4ROSAiz+SyvoGILBeR7SKySkTq5lhfSUSiRWSqvbejCmKM4T8/7GJJRDSP3RJgPzkknoG5/eH4dhgyT5ODUqWAnV5M/sAhY8wHwA6gl4hUtrGfK1Zj9m1Ac+AuEWmeY7O3gbnGmFbAJKwklN1/gDUFnUvZ997y/cz+PYqxXfx4tGeAvZ0uxMKn/eHkHhi2AJre7twglVIlgp02iC+BdBFpBHwE1AMW2NivA3DAGHPQGHMRWAjk/LOzObDC8f3K7OtFpB1QA/jVxrmUDUdOJzJtZSSD29Xl+b7N8p7sZ91kOOTIy+dPwpx+cGoPtBoCjXsXWbxKqeJlJ0FkGGPSgIHAVGPMU0AtG/vVAY5mex3tWJbdNsdxAQYAFUWkmoi4AP8PeNLGeZRN9auV46uHQnh9YEtcXPIpoFcnyJoGdOc3MKcvnDkIZTytBKGUKjXs9mK6CxgFfO9YVqaQzv8k0M3RhbYbEAOkAw8BPxpjovPbWUTGiUi4iISfOnWqkEK68fz053G+3mL9KAPreBU8h7RfV/jH+7BkLJw5BG7uMOwznfVNqVLGTi+me4AHgdeMMYdExA+YZ2O/GKzHUZnqOpZlMcYcw3EHISIVgDuNMXEi0hkIFZGHsOaeKCsi540xz+TYfwYwAyA4ODjntKgKWL3vFI8s3EKbepXp37oOrvndOWRKTYLfJ1vfZ6RCx8c1OShVCtkp970LeCTb60PAmzaOvQkIcCSUGGAYMDz7BiLiDZwxxmQAzwKzHOcYkW2bMUBwzuSgCrYp6gwPzAsnoHpFPhnd3l5yyEiHL++D6I3gXhE6jrdmhfML1SShVCljpxdTgIgsEZFdInIw86ug/RztFhOAX4DdwCJjzE4RmSQi/R2bdQf2isg+rAbp1676nahL7IiJZ+zsTdT28mTuvR3w8rTxVNAY+Olp2PM9lClv9Vjq8RwMnmO1SRzSDmVKlSZ2HjHNBl4C3gVuxnrkZGvmeWPMj8CPOZa9mO37JcCSAo4xB5hj53zqb+sOxFLRw41593XEu4K7zZ3ehU2fQIMu0P2Zv+8Y/LpaSSJms95FKFWKiDH5P7oXkQhjTDsR+dMY0zL7siKJ0Kbg4GATHh5e3GEUO2NMVvfV+MRUvMrZ7E+wbaE1G1zgIKuEhoutvwGUUtc5x+d5cG7r7HwKpDi6ne4XkQkiMgCr4ViVMKcSUrjzwzC2R8cB2E8OkSuteaR9Q+Gf0zQ5KKUAewniUaAcVkN1O2AkMNqZQakrF5+Yyt0zN7D7eAKp6VdQS/H4dvjibvBuYnVldbP5OEopdcOz04tpk+Pb81jtD6qYTV8dSau6XlkzvF1ISWPQ9N85eCqROWPb066BzTkd4o7AZ4PBoxKMWAweNicKUkqVCnZ6MS3LXntJRKqIyC9OjUrlq1VdLyYs2EJYZCwpaekM/Wg9+09e4JGejQgNuILie/MHWWMeRn4JXjkHuSulSjs7vZi8jTFxmS8cEwZVd15IqiAh/t5MHd6WCQu2MKx9PfaeSOCBrg159JbG9g6QmmxNFXr2ENz9NVRv5tyAlVLXJVu1mESkfuYLEWmAY/IgVXxC/L25q0M9pq2K5MFu/jx7u80P+Yx0+Or+v2eD873JuYEqpa5bdu4gngPWichqQIBQQGenL2Zv/LSHj9ce5L6b/PhswxE6+1fLapPIkzHwy//B7qVw638hcGD+2yulSjU7Ewb9DAQBX2CV7G5njNE2iGI0d30U01dH0rRmRZ65rWnW46awyNj8dwx7HzZMh04PQ+eHiyZYpdR1y+6I6FhjzPdYNZEK+BRSznTiXDJv/rSH6hXdmX9vR9xcXbLaJLZHx+e9459LYNkL0GIA9H616AJWSl23rnREVP+CN1HOkpyazrh5ERhg3r0dqVK+bNa6EH9vHuzmn/uOh9bA1w9aJTT+OV0HwimlbLnSTwob5UCVs8QlppKSms47Q9rQpGZFezud2AkLR0C1RtZAuDIezg1SKXXDKLCRWkS6GGN+d7xsl8syVURqennw/cSbCp7wJ1N8tDXWoWwFGLkEPKs4N0Cl1A3FzifN+5nfOOZtuGSZcr41+07xyOdbSLyYZj85JMVZyeHieSs5eNV1aoxKqRtPnncQjlndQgAfEXki26pKgKuzA1OWQ7EXmLBgM7Ure1JA4d2/paVYj5VOH4C7v4IaLZwao1LqxpTfI6ayWFVb3YDsD7zPAYOcGZSyJCSncv/ccFxdhI9HBVPe3cawlYwMq2z34XVw50ydv0EpddXy/MQxxqwGVovIHGPMYQBH2e8KxphzRRVgaZWRYXj8i60cir3AvHs7UK9qOXs7LnsBdn4NvSZBS83jSqmrZ+eB9usiUklEygM7gF0i8pST4yr1jpxJJOLwWV7s17zgEdKZ1n8A66dChwcg5JGCt1dKqXzYSRDNHXcM/wR+AvyAu50ZlAJf7/L89kQ3RnVuYG+HHV9ZZTSa9Yc+r4Noj2Sl1LWxkyDKiEgZrASx1BiTihbrc5qdx+L5YOUBjDFUq+CeNX3oJdZNtga/ZYpaZxXgq1zfMV2o9iFQSl07OwliOhAFlAfWOKq5ahuEE8SeT2Hc3Ajm/3GY+KTUvDesEwSLx1hJ4uRu+GyIVYjv1jd0IJxSqtDk2y3G0Sh9whhTJ9uyI8DNzg6stLmYlsFD8zcTez6FxQ92pnK5snlv7NcVBs+BRaMgPQ1SE+HOT6BZ3yKLVyl148v3DsIxMO7pHMuMMSbNqVGVQpO+38nGqDP8b1ArWtWtXPAO9TqBmydcTICgUdpjSSlV6Ow8YvpNRJ4UkXoiUjXzy+mRlSL7TyTw+cajPNCtIXe0sTn156JRkHDMapTe8/2lbRJKKVUI7EwYNNTxb/YJBAzQsPDDKZ0CalTk24e70KxWJXs7/Po87PsJWgyEwbOt5LB4jPXYSQfGKaUKSYEJwhjjVxSBlEbRZxPZfTyBXs1rEFjHy+ZOEbB+GtRqY7U7wN9tEjGbNUEopQqNnWquZYDxQOYnzyrgI0d3V3WVki6mM25uBNFnE1n7dA+8ypUpeKfzJ+GLkeBVB+7++tLurH5dNTkopQqVnUdMHwJlgGmO13c7lt3nrKBudMYYnlqyjd1/nWPW6Pb2kkN6qvUYKeks3PsrlNNmIKWUc9lJEO2NMa2zvV4hItucFVBpMG1VJN9vP84ztzXl5qbV7e306wtw+HdrIFytVs4NUCmlsNeLKV1EsuayFJGGQLrzQrqx7T5+jrd/3csdbWrzQFeb7fzbvoANH0Knh6DVEOcGqJRSDnbuIJ4CVorIQawpRxsA9zg1qhtY05oVeW9YW3o3r5F7GY2cjm+D7x6BBjdZFVqVUqqI5HkHISKPiUgHYDUQADwCTASaGGNW2jm4iPQRkb0ickBEnsllfQMRWS4i20VklYjUdSxvIyLrRWSnY93Qy49+fYlPTGXfiQREhP6ta+NRxka9pAunYeFIKFfN6qXkaqOtQimlCkl+j5jqApOBk8CvwDCgPlZNpgKJiCvwAXAb0By4S0Sa59jsbWCuMaYVMAl43bE8ERhljGkB9AEmi0hlO+ctKaavjiQsMhaAtPQMJny+mQHTfue93/bZO0B6Giy5B86fgKHzoIKPE6NVSqnL5ZkgjDFPGmNCgJrAs8AZrEdLO0Rkl41jdwAOGGMOGmMuAguBO3Js0xxY4fh+ZeZ6Y8w+Y8x+x/fHsJLUdfUJ2aquFxMWbCEsMpY3f97D2v2xYKC9n83eR8tfgUOrod87UKedc4NVSqlc2GmD8MSah9rL8XUM+NPGfnWAo9leRwMdc2yzDRgIvAcMACqKSDVjzOnMDRyPucoCkTlPICLjgHEA9evXtxFS0Qnx92bq8LbcPzecCynpuLu58PHoYHuT/+z4CsKmQPC90Hak84NVSqlc5JkgRGQG0AJIADYAYcA7xpizhXj+J4GpIjIGWAPEkK2HlIjUAuYBox2FAy9hjJkBzAAIDg4ucXNU1PLyJCXVCvu+UD97yeHETvj2YajXEfq84eQIlVIqb/m1QdQH3IG/sD64o4G4Kzh2DFAv2+u6jmVZjDHHjDEDjTFtgeccy+IARKQS8APwnDHmjys4b4lx+PQF3FyFsV18+Xzj0aw2iTwlnYWFI8C9EgyZC275lPxWSikny/MOwhjTR6x+mC2AEOBfQKCInAHWG2NeKuDYm4AAEfHDSgzDgOHZNxARb+CM4+7gWWCWY3lZ4GusBuwlV/XOitnqfSd5YtE2Zo1pT4i/N7c0r8GEBVuYOrxt7ncSGenw5f0QHw1jfoCKNYs+aKWUyqag+SCMMWYH8CPWfNS/A/7AowUd2DFnxATgF2A3sMgYs1NEJolIf8dm3YG9IrIPqAG85lg+BKv20xgR2er4anOlb664/LLzLx75fAsv9GuWlQwy2yS2R8fnvtOq1+HAMrjtTaifs6lGKaWKnhiT+6N7EXkE684hBEjFaoPI/PoztzaB4hQcHGzCw8OLOwyOxSVx23trqVfVky/Hh+DuZmO8w+7v4YsR0PZu6P8+2BlAp5RShUBEIowxwbmty68Xky+wGHjcGHPcGYHdaNLSM3hs4VbS0jN4/64ge8nh1F74+kGoHQS3v63JQSlVYuTXBvFEUQZyI5iy4gAbo84weWgb/LxtjCdMPmc1SpfxsAbDlfFwfpBKKWWTnXEQyob0DEN41BnuDKrLP9vamDY0I8O6czhzEEYvBa+6zg9SKaWugCaIQuLqIsy7tyOp6TabZta+DXt/gD5vgu9Nzg1OKaWugp1y3yofxhje+20/JxOScXURe0X49v0CK/8LrYZCxwecH6RSSl0FTRDXaPbvUbz72z5+3XnC3g6nI63xDjVbQr/J2iitlCqxNEFcgx0x8bzx0x5uaVaDER1t1IJKOW81Sru4wtD5ULac84NUSqmrpG0QV+l8ShoTP99C1fJleWtQq4In/zEGvn0IYvfCyK+gSoOiCVQppa6SJoir9O6yfRw+fYEF93eiSnkbNZN+fw92fWvNCud/s/MDVEqpa6QJ4ipN7NGIoPpV6NSwWu4brJsMdYLArytErrDmd/C9ybqTUEqp64C2QVyhk+eSuZiWQeVyZenbqlbeG9YJgsVjYPsiWDIWvOrBiV3WcqWUug7oHcQVSElL5545m/Cu4M6nYzvkv7FfVxj4MSwYDOJm3TkMnWctV0qp64DeQVyBN3/ay85j5xjZyWYD8/5frTLe6SnQYZwmB6XUdUUThE0r9pxg1u+HGBPiS6/mNQreYff3sGE6uHlA16chfCYcWuP8QJVSqpBogrDhr/hknly8nea1KvHMbU0L3iHuCHw1zhrvMGwB9HgOBs+x2iQ0SSilrhOaIGw4n5JGvSqevD+8bcGlNNJTYcm9kJEKd3wIjXpay/26WkkiZrPT41VKqcKgjdQ2NKpegW8e7lLwYDiAFa9C9EYYNBsCB166zq+rtkMopa4begeRj01RZ/j3ku0kXkyzlxz2/wa/T4Z291yeHJRS6jqjdxB5iEu8yKOfb6GsmwsZdsa2nTsOX4+D6i2gz+tOj08ppZxNE0QujDE8vWQ7p86n8NX4LlRwL+DHlJEOX94HqUkweDaU8SyaQJVSyon0EVMu5v1xmF93neDffZrSsq5XwTus/h8cXgd9/x/4NHF+gEopVQQ0QeSQdDGdKcv3072JD2O7+BW8w6E1sPpNaH0XtBnu/ACVUqqI6COmHDzLuvLV+C6Uc3fFxaWAhunzp6zJf6o1gtvfLpoAlVKqiOgdRDbr9sdijKF+tXJ4V3DPf+OMDPj6AUg6a41vcK9QJDEqpVRRKdUJYvrqSMIiYwH4dmsMI2du4H+/7GX66siCdw57DyKXWz2WagY6OVKllCp6pTpBtKrrxYQFW/h6SzTPfb2DxjUqsHDjEVoV1DB9ZAMs/w80/ycEjy2SWJVSqqiV6jaIEH9vpg5vy+hZGxHg5LkUpo0MIsTfO++dEs845neoC/2ngJ0BdEopdR0q1XcQYCWJUZ19uZhuGNW5Qf7JwRj4dgKcP2GNd/Cw0QVWKaWuU6U+QYRFxvL1lhge6dGI+RuOZLVJ5GrDR7D3B2te6Trtii5IpZQqBqU6QYRFxjJhwRamDm/LE72bMHV4WyYs2JJ7kojZDL8+D01uh07jiz5YpZQqYk5NECLSR0T2isgBEXkml/UNRGS5iGwXkVUiUjfbutEist/xNdoZ8W2Pjmfq8LZZj5Uy2yS2R8dfumFyPCy5BypUhzs+0HYHpVSpIMbYqUR3FQcWcQX2Ab2AaGATcJcxZle2bRYD3xtjPhWRHsA9xpi7RaQqEA4EAwaIANoZY87mdb7g4GATHh5e+G/EGCs57FoKY36ABp0L/xxKKVVMRCTCGBOc2zpn3kF0AA4YYw4aYy4CC4E7cmzTHFjh+H5ltvW3AsuMMWccSWEZ0MeJseYtYg7s/NqaFU6Tg1KqFHFmgqgDHM32OtqxLLttQObECQOAiiJSzea+zndiJ/z8DDS8Gbo8XuSnV0qp4lTcjdRPAt1EZAvQDYgB0u3uLCLjRCRcRMJPnTpVuJFdvGDNIe3hBQNngEtx/6iUUqpoOfNTLwaol+11XceyLMaYY8aYgcaYtsBzjmVxdvZ1bDvDGBNsjAn28fEp3Oh/fApi98PAj63GaaWUKmWcmSA2AQEi4iciZYFhwNLsG4iIt4hkxvAsMMvx/S9AbxGpIiJVgN6OZUVj6+ew9TPo+hQ07FZkp1VKqZLEaQnCGJMGTMD6YN8NLDLG7BSRSSLS37FZd2CviOwDagCvOfY9A/wHK8lsAiY5ljlf7H744V/QoAt0+3eRnFIppUoip3VzLWqF0s01NQk+uQXOHYPxv0Ol2oUTnFJKlVD5dXMt1cX6LvPLc3BiBwxfrMlBKVXqle6uOesmW1OGgjXWIXwmtBgAJ3flu5tSSpUGpTtB1AmyurJuWwRLHwHvJlbCqBNU3JEppVSxK92PmPy6wsBPYMFgEFe4cBKGzLWWK6VUKVe67yAAqvpBmfKQngLt79fkoJRSDpog4o+Cqxt0fdpqg8hsk1BKqVKudCeIQ2usNojBc6xifIPnWK81SSilVClPEDGbraSQ+VjJr6v1OmZzcUallFIlQulupL7pscuX+XXVdgillKK030EopZTKkyYIpZRSudIEoZRSKleaIJRSSuVKE4RSSqlc3TDlvkXkFHA4l1VeQLyNZd5ArBNCK0husRTVcezuU9B2ea2/kuUl6ZpA8V0XZ1+T/NaV9OtS0n9XrmWb4rwmDYwxuU/JaYy5ob+AGTaXhZeU+IrqOHb3KWi7vNZfyfKSdE2K87o4+5pcz9elpP+uXMs2JfWalIZHTN/ZXFZcCiuWqzmO3X0K2i6v9VeyvCRdEyi+6+Lsa5LfupJ+XUr678q1bFMir8kN84jpWolIuMljViVVPPSalEx6XUoeZ12T0nAHYdeM4g5AXUavScmk16Xkcco10TsIpZRSudI7CKWUUrnSBKGUUipXmiCUUkrlShNELkSkoYjMFJElxR2L+puI/FNEPhaRL0Skd3HHo0BEmonIdBFZIiLjizse9TcRKS8i4SLS72qPUWoShIjMEpGTIrIjx/I+IrJXRA6IyDMAxpiDxph7iyfS0uUKr8s3xpj7gQeBocURb2lwhddktzHmQWAI0KU44i0truS6OPwbWHQt5yw1CQKYA/TJvkBEXIEPgNuA5sBdItK86EMr1eZw5dflecd65RxzuIJrIiL9gR+AH4s2zFJnDjavi4j0AnYBJ6/lhKUmQRhj1gBncizuABxw3DFcBBYCdxR5cKXYlVwXsbwJ/GSM0XlhneRKf1eMMUuNMbcBI4o20tLlCq9Ld6ATMBy4X0Su6rO+dE85CnWAo9leRwMdRaQa8BrQVkSeNca8XizRlV65XhdgInAL4CUijYwx04sjuFIqr9+V7sBAwB29gygOuV4XY8wEABEZA8QaYzKu5uClPUHkyhhzGus5typBjDFTgCnFHYf6mzFmFbCqmMNQeTDGzLmW/UvNI6Y8xAD1sr2u61imipdel5JHr0nJ5NTrUtoTxCYgQET8RKQsMAxYWswxKb0uJZFek5LJqdel1CQIEfkcWA80EZFoEbnXGJMGTAB+AXYDi4wxO4szztJGr0vJo9ekZCqO66LF+pRSSuWq1NxBKKWUujKaIJRSSuVKE4RSSqlcaYJQSimVK00QSimlcqUJQimlVK40QahCISI1RWShiESKSISI/CgijQvhuN1F5HvH9/0zyxk75oZonm27SSJyy7Wez5lE5HwhH6+tiMx0fD9GRKYW5vGvloj45ixJncs2PiLyc1HFpK6OJgh1zUREgK+BVcYYf2NMO+BZoEZhnsdRNfQNx8t/YpU3zlz3ojHmt8I8X0kiIrnVTfs/rtPaVMaYU8BxEdE5JEowTRCqMNwMpGavrmqM2WaMWeso0f2WiOwQkT9FZChk3RmscsxEtkdEPnMkmswJUPaIyGasSqE4lo8RkakiEgL0B94Ska0i4i8ic0RkkGO7niKyxXG+WSLi7lgeJSKviMhmx7qmOd+I4xxficjPIrJfRP6Xbd35bN8PEpE5ju/niMiHIvKHiBx0vLdZIrI7c5ts+70rIjtFZLmI+DiW+TvOFyEiazPjchx3uohsAP6X4zgVgVbGmG25vAdfEVkhItsd56mf7Tx/ON77q7nd0Yg1C9kPIrLNcc0yr1d7EQlzLN8oIhUd51nr+HludlyXnMdzdVz/TY54Hsi2+hu0RHiJpglCFYZAICKPdQOBNkBrrFLdb4lILce6tsBjWHcCDYEuIuIBfAz8A2gH1Mx5QGNMGFa9maeMMW2MMZGZ6xz7zwGGGmNaYlUszj4VZqwxJgj4EHgyj5jbYM1Y1xIYKiL18tguuypAZ+BxR2zvAi2AliLSxrFNeSDcGNMCWA285Fg+A5jouPN6EpiW7bh1gRBjzBM5zhcM5PUY533gU2NMK+Az/r7LeA94z/Fzic5j3z7AMWNMa2NMIPCzWDV+vgAeNcZkXsckrMloejl+nkPJ/W7mXiDeGNMeaI81N4GfY104EJpHHKoE0AShnO0m4HNjTLox5gTWB2N7x7qNxphoR636rYAv0BQ4ZIzZb6w6MPOv8HxNHPvvc7z+FOiabf1Xjn8jHOfLzXJjTLwxJhlrVq4GNs77nSPeP4ETxpg/He9rZ7bzZGB90IL1vm4SkQpACLBYRLYCHwG1sh13sTEmPZfz1QJO5RFLZ2CB4/t5WNcgc/lix/cLcu7k8CfQS0TeFJFQY0w81s/0uDFmE4Ax5pyjBlAZ4GMR+dNx3NxmY+wNjHK8tw1ANSDAse4kUDuPOFQJoPNBqMKwExh0FfulZPs+naL5/5h5zvzOl1dc2QuXeeSxT0aO/TPyOY/B+iMtzhjTJo9tLuSxPCmXGK6ZMWafiAQBtwOvishyrPal3DwOnMC6O3QBknPZRrDujn7JZZ0H1vtQJZTeQajCsAJwF5FxmQtEpJWIhAJrsR7TuDqeuXcFNuZzrD2Ar4j4O17flcd2CUDFXJbvdezfyPH6bqy7lsJwQkSaiTV944Cr2N+FvxPpcGCdMeYccEhEBoPV4C8irW0cazfQKI91YVhln8F6xr/W8f0fwJ2O74fl3Mlx/tpAojFmPvAWEIT1M60lIu0d21QUq9HcC+vOIgPr5+yayyF/AcaLSBnHvo1FpLxjXWPyfkymSgBNEOqaOR6tDABuEaub607gdeAvrL8+twPbsBLJ08aYv/I5VjIwDvjB0Uid16TrC4GnHI3R/jn2vwfrkc2fWH/BF9bUpM8A32N9AB+/iv0vAB3E6gLaA5jkWD4CuFdEtmHdjRU4L7oxZg/W1Ku5JcmJwD0ish3rg/tRx/LHgCccyxsB8bns2xLY6Hgk9BLwqmOu46HA+44Yl2H99T8NGO1Y1pTc73Y+wXpMt9nxvj/i7zuqm4EfCnqvqvhouW+lrlMi8jiQYIz5xOb25YAkY4wRkWHAXcaYApORs4jIGuAOY8zZ4opB5U/bIJS6fn0IDL6C7dsBU0VEgDhgrDOCssPxuPEdTQ4lm95BKKWUypW2QSillMqVJgillFK50gShlFIqV5oglFJK5UoThFJKqVxpglBKKZWr/w9czCbLzDFr3AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot theoretical and PEPit (numerical) worst-case performance bounds as functions of the iteration count\n", "\n", "plt.plot(L/mus, theoretical_taus, '--x', label='Theoretical upper bound')\n", "plt.plot(L/mus, pepit_taus, '-x', label='PEPit worst-case bound')\n", "\n", "plt.semilogx()\n", "plt.legend()\n", "plt.xlabel('Condition number (log scale)')\n", "plt.ylabel('Worst-case guarantee')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "toc": { "base_numbering": 1, "nav_menu": { "height": "341px", "width": "386px" }, "number_sections": true, "sideBar": true, "skip_h1_title": true, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": { "height": "379px", "left": "1067.99px", "top": "518px", "width": "574px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 5 }