{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ ".. meta::\n", " :description: Using the responsible AI toolbox to build workflows." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Copyright (c) 2023 Massachusetts Institute of Technology \n", "> SPDX-License-Identifier: MIT \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Building Workflows for Cross Validation Training and Adversarial Robustness Analysis\n", "This notebook demonstrates how the to build experiment workflows for configurable, repeatable, and scalable (CRS) experimentation. Two basic workflows will be demonstrated in this tutorial:\n", "\n", "- Cross-Validation Workflow: Performs cross-validation training that logs accuracy and loss across data folds\n", "- Robustness Curve Workflow: Loads a trained model and assesses the impact of adversarial perturbations on the model's performance; the model's performance metric is plotted against the increasing \"severity\" of the perturbation\n", "\n", "Here, \"workflow\" has a precise meaning. In the parlance of [mushin](https://mit-ll-responsible-ai.github.io/responsible-ai-toolbox/ref_mushin.html) a workflow is an API for describing how we configure, launch, and post-process one or more tasks. These workflows leverage [Hydra](https://hydra.cc/) and [hydra-zen](https://github.com/mit-ll-responsible-ai/hydra-zen) so that they are highly configurable and so that each job launched by a workflow is self-documenting and reproducible. \n", "In this tutorial, we also make use of [PyTorch Lightning](https://www.pytorchlightning.ai/) to eliminate boilerplate code associated with training and testing a PyTorch model.\n", "\n", "## Getting Started\n", "\n", "\n", "We will install the rAI-toolbox and then we will create a Jupyter notebook in which we will complete this tutorial.\n", "\n", "\n", "### Installing `rai_toolbox`\n", "\n", "\n", "To install the toolbox (along with its `mushin` capabilities) in your Python environment, run the following command in your \n", "terminal:\n", "\n", "```console\n", "$ pip install rai-toolbox[mushin]\n", "```\n", "\n", "To verify that the toolbox is installed as-expected, open a Python console and try \n", "importing ``rai_toolbox``.\n", "\n", "```python\n", ">>> import rai_toolbox\n", "```\n", "\n", "You will also need to install scikit-learn; please follow [these instructions](https://scikit-learn.org/stable/install.html#installing-scikit-learn).\n", "\n", "\n", "## Opening a Jupyter notebook\n", "\n", "If you do not have Jupyter Notebook or Jupyter Lab installed in your Python environment, please follow [these instructions](https://jupyter.org/install).\n", "Now open a terminal on your computer and [start a notebook/lab session](http://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Jupyter_Notebooks.html).\n", "A file-viewer will open in an internet browser; pick a directory where you are okay with saving some PyTorch model weights. Create a notebook called `Building-Workflows.ipynb`. You can then follow along with this tutorial by copying, pasting, and running the code blocks below in the cells of your notebook.\n", "\n", "Note: you may also need to install the `ipywidgets` package in your Python environment to configure the notebook to display ipywidgets:\n", "\n", "```console\n", "$ pip install ipywidgets\n", "```\n", "\n", "## Imports" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "from typing import Optional, Tuple, Union\n", "\n", "import matplotlib.pyplot as plt\n", "import torch as tr" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Hydra and hydra-zen\n", "from hydra.core.config_store import ConfigStore\n", "from hydra_zen import MISSING, builds, instantiate, load_from_yaml, make_config\n", "\n", "# Lightning\n", "from pytorch_lightning import LightningModule, Trainer\n", "\n", "# sklearn and torch\n", "from sklearn.model_selection import StratifiedKFold\n", "from torch import Tensor, nn\n", "from torch.optim import Optimizer\n", "from torch.utils.data import DataLoader, Subset\n", "from torchmetrics import Accuracy\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "\n", "# rAI-toolbox\n", "from rai_toolbox._typing import Partial\n", "from rai_toolbox.mushin import load_from_checkpoint\n", "from rai_toolbox.mushin.lightning import MetricsCallback\n", "from rai_toolbox.mushin.workflows import (\n", " MultiRunMetricsWorkflow,\n", " RobustnessCurve,\n", " multirun,\n", ")\n", "\n", "from rai_toolbox.optim import L2ProjectedOptim, LinfProjectedOptim\n", "from rai_toolbox.perturbations import gradient_ascent" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Experiment Functions and Classes\n", "\n", "Here we define two Neural Network models, a fully linear neural network and a convolutional neural network." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "class LinearModel(nn.Module):\n", " def __init__(self):\n", " super().__init__()\n", " self.model = nn.Sequential(\n", " nn.Flatten(1),\n", " nn.Linear(28 * 28, 256),\n", " nn.ReLU(),\n", " nn.Linear(256, 128),\n", " nn.ReLU(),\n", " nn.Linear(128, 64),\n", " nn.ReLU(),\n", " nn.Linear(64, 10),\n", " )\n", "\n", " def forward(self, x):\n", " return self.model(x)\n", "\n", "class ConvModel(nn.Module):\n", " def __init__(self):\n", " super().__init__()\n", " self.model = nn.Sequential(\n", " nn.Conv2d(1, 32, 5, padding=\"same\"),\n", " nn.BatchNorm2d(32),\n", " nn.ReLU(),\n", " nn.MaxPool2d(3),\n", " nn.Conv2d(32, 32, 3, padding=\"same\"),\n", " nn.BatchNorm2d(32),\n", " nn.ReLU(),\n", " nn.MaxPool2d(3),\n", " nn.Conv2d(32, 32, 3, padding=\"same\"),\n", " nn.BatchNorm2d(32),\n", " nn.ReLU(),\n", " nn.Conv2d(32, 10, 3),\n", " nn.Flatten(1),\n", " )\n", "\n", " def forward(self, x):\n", " return self.model(x)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next lets define a function that takes the `MNIST` dataset and splits the data into training and validation sets using SciKit-Learn's [StratifiedKFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html). This allows us to split the dataset into \"folds\" and select the fold for each experiment." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def split_dataset(\n", " dataset: MNIST, n_splits: int, fold: int, random_state: int = 49\n", ") -> Tuple[Subset, Subset]:\n", " \"\"\"Provide training and validation splits using `sklearn.model_selection.StratifiedKfold`\"\"\"\n", "\n", " kfold = StratifiedKFold(n_splits=n_splits, shuffle=True, random_state=random_state)\n", " train_indices, val_indices = list(\n", " kfold.split(range(len(dataset)), dataset.targets)\n", " )[fold]\n", " return Subset(dataset, train_indices), Subset(dataset, val_indices)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now define the [LightningModule](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.core.LightningModule.html#pytorch_lightning.core.LightningModule) for training and testing.\n", "This describes how we:\n", "\n", "- Load our data\n", "- Process a batch of data with our model (both with and without adversarial perturbations)\n", "- Update our model's parameters during training\n", "\n", "Note that we specifically design this lightning module to log the following metrics:\n", "\n", "- Loss and accuracy for cross-validation training\n", "- Adversarial loss, adversarial accuracy, and clean accuracy for robustness analysis\n", "\n", "These metrics will be saved during each of our runs, and we will load and aggregate these metrics to analyze our results. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "class StandardModule(LightningModule):\n", " def __init__(\n", " self,\n", " *,\n", " model: nn.Module,\n", " dataset: MNIST,\n", " optimizer: Optional[Partial[Optimizer]] = None,\n", " perturber=None,\n", " fold: int = 0,\n", " n_splits: int = 5,\n", " batch_size: int = 100,\n", " num_workers: int = 4,\n", " ) -> None:\n", " super().__init__()\n", " self.dataset = dataset\n", " self.optimizer = optimizer\n", " self.criterion = nn.CrossEntropyLoss()\n", " self.model = model\n", " self.perturber = perturber\n", " self.n_splits = n_splits\n", " self.fold = fold\n", " self.batch_size = batch_size\n", " self.num_workers = num_workers\n", "\n", " # Metrics\n", " self.acc_metric = Accuracy(task=\"multiclass\", num_classes=10)\n", " if self.perturber:\n", " self.clean_acc_metric = Accuracy(task=\"multiclass\", num_classes=10)\n", "\n", " def forward(self, data: Tensor) -> Tensor:\n", " return self.model(data)\n", "\n", " def train_dataloader(self) -> DataLoader:\n", " train_dataset, _ = split_dataset(self.dataset, self.n_splits, self.fold)\n", " return DataLoader(\n", " train_dataset,\n", " batch_size=self.batch_size,\n", " num_workers=self.num_workers,\n", " shuffle=True,\n", " )\n", "\n", " def val_dataloader(self) -> DataLoader:\n", " _, val_dataset = split_dataset(self.dataset, self.n_splits, self.fold)\n", " return DataLoader(\n", " val_dataset, batch_size=self.batch_size, num_workers=self.num_workers\n", " )\n", "\n", " def test_dataloader(self) -> DataLoader:\n", " return DataLoader(\n", " self.dataset, batch_size=self.batch_size, num_workers=self.num_workers\n", " )\n", "\n", " def configure_optimizers(self) -> Optional[Optimizer]:\n", " if self.optimizer:\n", " return self.optimizer(self.model.parameters())\n", " return None\n", "\n", " def _step(self, batch, stage: str) -> Tensor:\n", " data_orig, target = batch\n", "\n", " if self.perturber:\n", " with tr.no_grad():\n", " output = self.model(data_orig)\n", " loss = self.criterion(output, target)\n", " acc = self.clean_acc_metric(output, target)\n", " self.log(f\"{stage}_clean_accuracy\", acc)\n", " \n", " inference_tensors = tr.is_inference_mode_enabled()\n", " with tr.inference_mode(mode=False), tr.enable_grad():\n", " if inference_tensors:\n", " # we need to clone in order to support grad mode\n", " data_orig = data_orig.clone()\n", " target = target.clone()\n", "\n", " data, adv_loss = self.perturber(\n", " model=self.model, data=data_orig, target=target\n", " )\n", " self.log(f\"{stage}_adversarial_loss\", adv_loss.mean().item())\n", "\n", " else:\n", " data = data_orig\n", "\n", " output = self.model(data)\n", " loss = self.criterion(output, target)\n", " acc = self.acc_metric(output, target)\n", " self.log(f\"{stage}_loss\", loss)\n", " self.log(f\"{stage}_accuracy\", acc)\n", " return loss\n", "\n", " def training_step(self, batch, batch_idx) -> Tensor:\n", " return self._step(batch, \"train\")\n", "\n", " def validation_step(self, batch, batch_idx) -> Tensor:\n", " return self._step(batch, \"val\")\n", "\n", " def test_step(self, batch, batch_idx) -> Tensor:\n", " return self._step(batch, \"test\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configuring our experiments with hydra-zen\n", "\n", "Now we use `hydra-zen` to create \"configs\" for all of the components of our experiments.\n", "Each config describes an interface and/or object in our experiment that we want to be able to modify from run to run.\n", "They will also serve to make our work self-documenting and reproducible." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "Augmentations = builds(\n", " transforms.Compose,\n", " [builds(transforms.RandomCrop, size=28, padding=4), builds(transforms.ToTensor)],\n", ")\n", "TrainDataset = builds(\n", " MNIST, root=\"${data_dir}\", train=True, transform=Augmentations, download=True\n", ")\n", "TestDataset = builds(\n", " MNIST,\n", " root=\"${data_dir}\",\n", " train=False,\n", " transform=builds(transforms.ToTensor),\n", " download=True,\n", ")\n", "ConvModelCfg = builds(ConvModel)\n", "LinearModelCfg = builds(LinearModel)\n", "Optim = builds(tr.optim.SGD, lr=0.1, zen_partial=True)\n", "\n", "\n", "L2PGD = builds(L2ProjectedOptim, zen_partial=True)\n", "LinfPGD = builds(LinfProjectedOptim, zen_partial=True)\n", "\n", "\n", "def lr_for_pgd(epsilon, num_steps):\n", " return 2.5 * epsilon / num_steps\n", "\n", "\n", "Perturber = builds(\n", " gradient_ascent,\n", " optimizer=\"${optimizer}\",\n", " epsilon=\"${epsilon}\",\n", " steps=\"${steps}\",\n", " lr=builds(lr_for_pgd, \"${epsilon}\", \"${steps}\"),\n", " zen_partial=True,\n", " populate_full_signature=True,\n", ")\n", "\n", "PLModule = builds(\n", " StandardModule,\n", " model=\"${model}\",\n", " fold=\"${fold}\",\n", " n_splits=\"${n_splits}\",\n", " dataset=TrainDataset,\n", " optimizer=Optim,\n", " perturber=\"${perturber}\",\n", " populate_full_signature=True,\n", ")\n", "\n", "\n", "EvalPLModule = builds(\n", " StandardModule,\n", " model=\"${model}\",\n", " dataset=TestDataset,\n", " perturber=\"${perturber}\",\n", " populate_full_signature=True,\n", ")\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We configure our trainer to use `MetricsCallback`, which will instruct PyTorch Lightning to automatically save our logged metrics as a dictionary in a file named \"fit_metrics.pt\" and \"test_metrics.pt\" for training and evaluation, respectively." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "TrainerCfg = builds(\n", " Trainer,\n", " max_epochs=10,\n", " accelerator=\"auto\",\n", " devices=1,\n", " enable_progress_bar=False,\n", " enable_model_summary=False,\n", " callbacks=[builds(MetricsCallback)],\n", " populate_full_signature=True,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we use Hydra's [ConfigStore](https://hydra.cc/docs/tutorials/structured_config/config_store/) API to create named configuration groups that can be specified/swapped when we run our workflow.\n", "Let's make it easy to swap both models and optimizers by-name." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "cs = ConfigStore.instance()\n", "cs.store(name=\"cnn\", group=\"model\", node=ConvModelCfg)\n", "cs.store(name=\"linear\", group=\"model\", node=LinearModelCfg)\n", "cs.store(name=\"l2pgd\", group=\"optimizer\", node=L2PGD)\n", "cs.store(name=\"linfpgd\", group=\"optimizer\", node=LinfPGD)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cross Validation Workflow\n", "\n", "With all the configurations in place we can now define our first experiment workflow: train multiple models on `MNIST` data using cross-validation. First define the main experiment configuration:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "import platform\n", "\n", "Config = make_config(\n", " defaults=[\n", " \"_self_\",\n", " {\"model\": \"linear\"},\n", " ],\n", " data_dir=Path.home() / \".torch/data\",\n", " model=MISSING,\n", " module=PLModule,\n", " trainer=TrainerCfg,\n", " perturber=None,\n", " fold=0,\n", " n_splits=5,\n", ")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create `CrossValWorkflow` by inheriting from [MultiRunMetricsWorkflow](https://mit-ll-responsible-ai.github.io/responsible-ai-toolbox/generated/rai_toolbox.mushin.MultiRunMetricsWorkflow.html) to train a given model for a given a cross validation dataset (fold). The task function simply runs PyTorch Lightning's [Trainer.fit](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.trainer.Trainer.html#pytorch_lightning.trainer.trainer.Trainer) and returns the metrics saved from [MetricsCallback](https://mit-ll-responsible-ai.github.io/responsible-ai-toolbox/generated/rai_toolbox.mushin.MetricsCallback.html). To run this workflow simply define the number of cross validation splits to use via `n_splits`." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "class CrossValWorkFlow(MultiRunMetricsWorkflow):\n", " @staticmethod\n", " def task(trainer: Trainer, module: LightningModule):\n", " trainer.fit(module)\n", " \n", " # Loads & returns a dictionary of metrics logged by PyTorch Lightning\n", " return tr.load(\"fit_metrics.pt\")\n", "\n", " def run(self, n_splits: int, **run_kwargs):\n", " fold = multirun(range(n_splits))\n", " super().run(n_splits=n_splits, fold=fold, **run_kwargs)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now run the workflow by defining the requried `n_splits` and the models (names defined in the [ConfigStore](https://hydra.cc/docs/tutorials/structured_config/config_store/) above). Additionally we define the working directory of the experiment by setting `hydra.sweep.dir` configuration via `overrides`." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:08:46,529][HYDRA] Launching 4 jobs locally\n", "[2023-01-27 13:08:46,529][HYDRA] \t#0 : n_splits=2 fold=0 model=linear\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py:74: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.\n", "See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.\n", " ret = run_job(\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/lightning_fabric/plugins/environments/slurm.py:166: PossibleUserWarning: The `srun` command is available on your system but is not used. HINT: If your intention is to run Lightning on SLURM, prepend your python command with `srun` like so: srun python /home/justin_goodwin/.conda/envs/rai_toolbox/lib/pyt ...\n", " rank_zero_warn(\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py:67: UserWarning: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `pytorch_lightning` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default\n", " warning_cache.warn(\n", "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n", "`Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:09:20,658][HYDRA] \t#1 : n_splits=2 fold=0 model=cnn\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py:74: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.\n", "See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.\n", " ret = run_job(\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/lightning_fabric/plugins/environments/slurm.py:166: PossibleUserWarning: The `srun` command is available on your system but is not used. HINT: If your intention is to run Lightning on SLURM, prepend your python command with `srun` like so: srun python /home/justin_goodwin/.conda/envs/rai_toolbox/lib/pyt ...\n", " rank_zero_warn(\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n", "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n", "`Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:09:56,220][HYDRA] \t#2 : n_splits=2 fold=1 model=linear\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py:74: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.\n", "See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.\n", " ret = run_job(\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/lightning_fabric/plugins/environments/slurm.py:166: PossibleUserWarning: The `srun` command is available on your system but is not used. HINT: If your intention is to run Lightning on SLURM, prepend your python command with `srun` like so: srun python /home/justin_goodwin/.conda/envs/rai_toolbox/lib/pyt ...\n", " rank_zero_warn(\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n", "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n", "`Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:10:28,555][HYDRA] \t#3 : n_splits=2 fold=1 model=cnn\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py:74: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.\n", "See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.\n", " ret = run_job(\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/lightning_fabric/plugins/environments/slurm.py:166: PossibleUserWarning: The `srun` command is available on your system but is not used. HINT: If your intention is to run Lightning on SLURM, prepend your python command with `srun` like so: srun python /home/justin_goodwin/.conda/envs/rai_toolbox/lib/pyt ...\n", " rank_zero_warn(\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n", "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n", "`Trainer.fit` stopped: `max_epochs=10` reached.\n" ] } ], "source": [ "kfold_task = CrossValWorkFlow(Config)\n", "kfold_task.run(\n", " n_splits=2,\n", " model=multirun([\"linear\", \"cnn\"]),\n", " overrides=[\n", " \"hydra.sweep.dir=outputs/cross_validation/${now:%Y-%m-%d}/${now:%H-%M-%S}\"\n", " ],\n", ")\n", "## You can load previous experiments\n", "# kfold_task = CrossValWorkFlow().load_from_dir(\"outputs/cross_validation/2022-05-11/14-42-14\", metrics_filename=\"fit_metrics.pt\")\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PosixPath('/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46')" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "kfold_task.working_dir" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the task is finished we can load in the metrics to an [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) using the `to_xarray` method.\n", "This xarray dataset thus stores all of the metrics that were saved/returned by the tasks that we ran." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:         (fold: 2, model: 2, epoch: 10)\n",
       "Coordinates:\n",
       "  * fold            (fold) int64 0 1\n",
       "  * model           (model) <U6 'linear' 'cnn'\n",
       "  * epoch           (epoch) int64 0 1 2 3 4 5 6 7 8 9\n",
       "Data variables:\n",
       "    train_loss      (fold, model, epoch) float64 1.8 0.9345 ... 0.09954 0.01555\n",
       "    train_accuracy  (fold, model, epoch) float64 0.47 0.68 0.83 ... 1.0 0.97 1.0\n",
       "    val_loss        (fold, model, epoch) float64 1.894 1.108 ... 0.07693 0.04691\n",
       "    val_accuracy    (fold, model, epoch) float64 0.3434 0.6266 ... 0.9743 0.9844\n",
       "Attributes:\n",
       "    n_splits:  2
" ], "text/plain": [ "\n", "Dimensions: (fold: 2, model: 2, epoch: 10)\n", "Coordinates:\n", " * fold (fold) int64 0 1\n", " * model (model) " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots()\n", "for name, g in xdata.groupby(\"model\"):\n", " g.val_accuracy[:, 0].plot.line(x=\"epoch\", label=f\"{name} (fold)\", ax=ax)\n", " g.val_accuracy[:, 0].mean(\"fold\").plot.line(\n", " x=\"epoch\", label=name, ax=ax, linestyle=\"--\", color=\"k\"\n", " )\n", "plt.legend()\n", "ax.set_title(\"Cross Validation Performance\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Robustness Curve\n", "\n", "Now that we have trained some models let's evalulate robustness of each model to adversarial perturbations. To create a configuration and workflow we need to provide the following for our task:\n", "\n", "- The experiment directory containing the experiment parameters and model checkpoint for training\n", "- The perturbation type and perturbation magnitude (`epsilon`)\n", "\n", "Below we define a function that extracts the configuration of the model defined by in the experiment YAML file while the checkpoint is found within the training experiment directory. This makes sure we load the correct model configuration and checkpoint for testing." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "def load_model_from_experiment(job_dir: Union[Path, str]):\n", " # Load configuration for experiment\n", " job_dir = Path(job_dir)\n", " exp_cfg = job_dir / \".hydra/config.yaml\"\n", " assert exp_cfg.exists()\n", " exp_cfg = load_from_yaml(exp_cfg)\n", " model = instantiate(exp_cfg.model)\n", "\n", " # get path to model checkpoint in experiment directory\n", " # and load to model\n", " ckpts = list(job_dir.glob(\"**/*.ckpt\"))\n", " assert len(ckpts) == 1\n", " ckpt = ckpts[0]\n", " load_from_checkpoint(\n", " model,\n", " ckpt=ckpt,\n", " weights_key=\"state_dict\",\n", " weights_key_strip=\"model.\",\n", " )\n", "\n", " return model\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can define our experiment configuration file." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "TestConfig = make_config(\n", " defaults=[\n", " \"_self_\",\n", " {\"optimizer\": \"l2pgd\"},\n", " ],\n", " data_dir=Path.home() / \".torch/data\",\n", " job_dir=MISSING, # will be auto-populated by our workflow\n", " trainer=TrainerCfg,\n", " module=EvalPLModule,\n", " model=builds(load_model_from_experiment, job_dir=\"${job_dir}\"),\n", " perturber=Perturber,\n", " optimizer=MISSING,\n", " epsilon=0.0,\n", " steps=10,\n", ")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The workflow inherits from [RobustnessCurve](https://mit-ll-responsible-ai.github.io/responsible-ai-toolbox/generated/rai_toolbox.mushin.RobustnessCurve.html). Other than the basic task function to use [Trainer.test](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.trainer.Trainer.html#pytorch_lightning.trainer.trainer.Trainer) and return the test metrics saved by [MetricsCallback](https://mit-ll-responsible-ai.github.io/responsible-ai-toolbox/generated/rai_toolbox.mushin.MetricsCallback.html), this workflow extends the basic interface for `RobustnessCurve` to:\n", "\n", "- Load and correlate the training experiment parameters model and fold to the robustness curve experiments\n", "- Update the [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) with the training experiment parameters" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "class ExperimentRobustnessCurve(RobustnessCurve):\n", " @staticmethod\n", " def task(trainer: Trainer, module: LightningModule):\n", " trainer.test(module)\n", " \n", " # Loads & returns a dictionary of metrics logged by PyTorch Lightning\n", " return tr.load(\"test_metrics.pt\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Run the experiment by first defining the main experiment directory for each cross-validation experiment and providing the `epsilon` values for the calculating the $\\ell_2$ adverarial perturbations." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:05,446][HYDRA] Launching 12 jobs locally\n", "[2023-01-27 13:11:05,447][HYDRA] \t#0 : epsilon=0 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py:74: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.\n", "See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.\n", " ret = run_job(\n", "/home/justin_goodwin/.conda/envs/rai_toolbox/lib/python3.10/site-packages/lightning_fabric/plugins/environments/slurm.py:166: PossibleUserWarning: The `srun` command is available on your system but is not used. HINT: If your intention is to run Lightning on SLURM, prepend your python command with `srun` like so: srun python /home/justin_goodwin/.conda/envs/rai_toolbox/lib/pyt ...\n", " rank_zero_warn(\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:05,602][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/0/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.9595000147819519\n", " test_adversarial_loss 0.12619434297084808\n", " test_clean_accuracy 0.9595000147819519\n", " test_loss 0.12619434297084808\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:07,343][HYDRA] \t#1 : epsilon=0 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:07,500][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/1/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.9868999719619751\n", " test_adversarial_loss 0.037374187260866165\n", " test_clean_accuracy 0.9868999719619751\n", " test_loss 0.037374187260866165\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:09,931][HYDRA] \t#2 : epsilon=0 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:10,091][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/2/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.9621999859809875\n", " test_adversarial_loss 0.12058829516172409\n", " test_clean_accuracy 0.9621999859809875\n", " test_loss 0.12058829516172409\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:11,825][HYDRA] \t#3 : epsilon=0 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:11,981][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/3/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.9898999929428101\n", " test_adversarial_loss 0.02972579002380371\n", " test_clean_accuracy 0.9898999929428101\n", " test_loss 0.02972579002380371\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:14,394][HYDRA] \t#4 : epsilon=1 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:14,549][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/4/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.22789999842643738\n", " test_adversarial_loss 3.348794937133789\n", " test_clean_accuracy 0.9595000147819519\n", " test_loss 3.348794937133789\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:16,295][HYDRA] \t#5 : epsilon=1 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:16,452][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/5/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.09929999709129333\n", " test_adversarial_loss 4.624772548675537\n", " test_clean_accuracy 0.9868999719619751\n", " test_loss 4.624772071838379\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:18,874][HYDRA] \t#6 : epsilon=1 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:19,029][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/6/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.2484000027179718\n", " test_adversarial_loss 3.2621147632598877\n", " test_clean_accuracy 0.9621999859809875\n", " test_loss 3.2621147632598877\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:20,770][HYDRA] \t#7 : epsilon=1 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:20,926][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/7/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.19189999997615814\n", " test_adversarial_loss 3.5013301372528076\n", " test_clean_accuracy 0.9898999929428101\n", " test_loss 3.5013301372528076\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:23,246][HYDRA] \t#8 : epsilon=2 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:23,413][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/0/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/8/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.004999999888241291\n", " test_adversarial_loss 10.608979225158691\n", " test_clean_accuracy 0.9595000147819519\n", " test_loss 10.608978271484375\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:25,194][HYDRA] \t#9 : epsilon=2 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:25,350][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/1/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/9/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.0\n", " test_adversarial_loss 16.613271713256836\n", " test_clean_accuracy 0.9868999719619751\n", " test_loss 16.6132755279541\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:27,776][HYDRA] \t#10 : epsilon=2 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:27,933][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/2/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/10/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.00559999980032444\n", " test_adversarial_loss 10.573102951049805\n", " test_clean_accuracy 0.9621999859809875\n", " test_loss 10.573102951049805\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", "[2023-01-27 13:11:29,668][HYDRA] \t#11 : epsilon=2 job_dir=/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[2023-01-27 13:11:29,853][rai_toolbox.mushin._utils][INFO] - Loading model checkpoint from /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/cross_validation/2023-01-27/13-08-46/3/lightning_logs/version_0/checkpoints/epoch=9-step=3000.ckpt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Missing logger folder: /home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04/11/lightning_logs\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " Test metric DataLoader 0\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " test_accuracy 0.0\n", " test_adversarial_loss 14.753412246704102\n", " test_clean_accuracy 0.9898999929428101\n", " test_loss 14.753414154052734\n", "────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n" ] } ], "source": [ "robustness_curve_task = ExperimentRobustnessCurve(TestConfig)\n", "robustness_curve_task.run(\n", " epsilon=[0, 1, 2],\n", " target_job_dirs=kfold_task.multirun_working_dirs,\n", " overrides=[\n", " \"hydra.sweep.dir=outputs/robustness_curves/${now:%Y-%m-%d}/${now:%H-%M-%S}\"\n", " ],\n", ")\n", "\n", "## You can load an old experiment\n", "# robustness_curve_task = ExperimentRobustnessCurve().load_from_dir(\"outputs/robustness_curves/2022-05-11/14-45-02\")\n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PosixPath('/home/justin_goodwin/projects/raiden/rai_toolbox/docs/source/tutorials/outputs/robustness_curves/2023-01-27/13-11-04')" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "robustness_curve_task.working_dir" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Extract the [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html)." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:                (epsilon: 3, job_dir: 4)\n",
       "Coordinates:\n",
       "  * epsilon                (epsilon) int64 0 1 2\n",
       "  * job_dir                (job_dir) <U117 '/home/justin_goodwin/projects/rai...\n",
       "    fold                   (job_dir) int64 0 0 1 1\n",
       "    model                  (job_dir) <U6 'linear' 'cnn' 'linear' 'cnn'\n",
       "Data variables:\n",
       "    test_clean_accuracy    (epsilon, job_dir) float64 0.9595 0.9869 ... 0.9899\n",
       "    test_adversarial_loss  (epsilon, job_dir) float64 0.1262 0.03737 ... 14.75\n",
       "    test_loss              (epsilon, job_dir) float64 0.1262 0.03737 ... 14.75\n",
       "    test_accuracy          (epsilon, job_dir) float64 0.9595 0.9869 ... 0.0
" ], "text/plain": [ "\n", "Dimensions: (epsilon: 3, job_dir: 4)\n", "Coordinates:\n", " * epsilon (epsilon) int64 0 1 2\n", " * job_dir (job_dir) " ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGwCAYAAABLvHTgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAADRfElEQVR4nOzdeVxUVRvA8d+dYd9lR0EWEUFFQUUqLfc0s0XcMNdX3NLS1Mw1t3IpS81yqaw0cU2kTEtzySwVzX0XRRDZZFEZ9mXmvn9MTBIuUMCInu/7mc873Dn33udM6H289znnSLIsywiCIAiCIDxmFPoOQBAEQRAEoSqIJEcQBEEQhMeSSHIEQRAEQXgsiSRHEARBEITHkkhyBEEQBEF4LIkkRxAEQRCEx5JIcgRBEARBeCwZ6DsAfdJoNCQlJWFpaYkkSfoORxAEQRCEcpBlmaysLGrXro1Ccf/7NU90kpOUlISbm5u+wxAEQRAE4V+4ceMGrq6u9/38iU5yLC0tAe2XZGVlpedoBEEQBEEoD5VKhZubm+46fj9PdJJT8ojKyspKJDmCIAiCUMM8rNREFB4LgiAIgvBYEkmOIAiCIAiPJZHkCIIgCILwWHqia3IEQRCeBGq1mqKiIn2HIQjlZmhoiFKp/M/HEUmOIAjCY0qWZVJSUrhz546+QxGECrOxscHZ2fk/zWMnkhxBEITHVEmC4+joiJmZmZj0VKgRZFkmNzeX1NRUAFxcXP71sUSSIwiC8BhSq9W6BMfOzk7f4QhChZiamgKQmpqKo6Pjv350pdfC4wMHDtC1a1ccHByQJAlJkli5cmW59l26dCkNGzbE2NgYR0dH/ve//5GSklLFEQuCINQMJTU4ZmZmeo5EEP6dkt/d/1JPptck58SJE+zevRtbW9sK7Td16lTGjh3LxYsXcXd3Jzs7m9WrV9OmTRtycnKqKFpBEISaRzyiEmqqyvjd1WuSM2DAAFQqFbt27Sr3PikpKSxcuBCACRMmEB0dTVRUFJIkER0d/cA7QQUFBahUqlKvSqdRQ+zvcHaL9v816so/hyAIgiAID6XXJMfOzk733K289u7dS3FxMQA9evQAoEmTJnh7ewM8MGGaP38+1tbWulelL855YRvyksawphtEhMGabtqfL2yr3PMIgiAIgvBQNW4ywBs3bujeOzo66t47OTkBEB8ff999p0yZQmZmpu5197H+swvbkDcPRFYlldosq5KQNw8UiY4gCDWWWiNzOCaDH04lcjgmA7VG1ndI1eLSpUsolUq8vLxQq/++K7927Vp8fHwwNDREkiRWr15d7mOW1J8+aJ/9+/fr2u3fv//fd+A/yMzM1K3rmJGRoZcYKkONS3Jk+d5/uEq2P+gZnrGxse4/WqUuyqlRk/fjRGRZLvOFKv6KLe/HieLRlSAINc7Oc8m0/mAffb+MYuzGU/T9MorWH+xj57lkfYdW5WbNmoVGo2HMmDG60T2pqamEhYVx5coVnJycCA4OxsHBQc+RVtz69etp1qwZpqam2Nra0rNnT65cuaL73NramrCwMLKysvjoo4/0GOl/U+OSnLp16+re37x5U/e+ZDx9pT+CKgd13EFM81JQ3JVfaWSZnEJt4qWQwDQvBXXcwWqPTRAE4d/aeS6Z18NPkJyZX2p7SmY+r4efeKwTndTUVCIiIlAoFPTt21e3PTo6WjfaZ+fOnURFRfHiiy/qK8x/5YsvvqBfv36cPHkSFxcX1Go1ERERtGrViqSkv59G9OvXD4BVq1bV2BmzH/kkx9fXF19fXz777DMAOnTogIGBdnqfLVu2AHDq1CmuXr0KQJcuXao9xphrMaV+vp0n89KGPEIj8tDcdefpn+0EQRCqkyzL5BYWl+uVlV/EzG3nude985Jts7ZdICu/qFzHu99d+PvFuXz5cgIDAzE1NcXS0pKWLVty6tQpAAYPHowkSbRt25bPPvsMDw8PLC0t6datW6mpRNq2bYskSQwYMICZM2fi4uJCrVq16N+/P1lZWQ+MYcuWLRQXF9OyZUtdOcSsWbN49tlndW38/f1LPVLatm0brVu3xsLCAlNTU5o1a8bXX3/90P5u3rwZLy8vTE1N6dq1K4mJiWXapKSk0K9fP1xcXDAyMsLBwYG2bduyY8cOXRsPDw8kSWLw4MH3PVdBQQFTp04FtHWt165d4+LFi1haWpKWlsb8+fN1bVu0aIGzszPp6ens2bPnof14FOl1MsCtW7fyzjvv6AqJAWbMmMFHH31EcHAw69at4/LlywCkp6cD4OzszMSJE5k/fz6LFy9m+/btJCQkIMsy9evXZ8SIEdXej1TZBp+7fo67o2HvtWIK1PD+gUJmtDG+ZztBEITqlFekpuGM8o9mfRAZSFHl4z/rl3K1vzCnM2ZG5bvkjBkzRvcPWzs7O5ydnTl9+jRxcXEEBATo2h06dIgjR47g5uZGdnY2O3bsYMKECaxbt67U8TZt2oSJiQn29vakpKSwbt063N3dmTt37n1j+P333wFo2bKlbpurqyt+fn5cvHgRgICAAF0ZRHh4OAMGDAC0NaImJiacPHmSsLAwkpOTmTZt2j3Pc+rUKfr27YtGo8Ha2pro6Oh7XsdGjRpFZGQkFhYW+Pv7k5aWxoEDB2jbtm2F7iQdO3ZMV2NTMnindu3aPPXUU+zevbvM4J2WLVuybds2fv/9d1544YVyn+dRodc7OSqVipiYGK5fv67blpaWRkxMzD0z2RJz585lyZIl+Pr6EhcXh7m5OYMGDeLAgQOYm5tXR+ilKD1akSTbUlKLF+iiZGU3EwBm7S9gR3QRSbItSo9W1R6bIAhCTRIXF8eyZcsACAkJISkpiXPnzpGQkEDz5s1LtVWr1Rw+fJjo6Gi6d+8OaEfg/pOJiQkXL17k6tWrtGjR4r7t7lZSn+Lh4aHbNnToUJYvX677OTIykqioKJo1a6ZLYoKDg7l+/TqxsbG6mObOnUtubu49z/Pxxx/rEpzLly9z9epVQkJC7hvPp59+yvHjx4mPjycxMZHQ0FBdm3r16tGgQYMHLoNQ0cE77u7upc5f0+j1Ts7gwYMfeFsN7l1oLEkSY8eOZezYsVUUWcW0rOfANMOhzCv6EI2srcEZHGDE0UQ1K44V0T8yj2+HW9HVy17foQqC8AQzNVRyYU7ncrU9GnuLwd/8+dB2q/8XREvPh0/oampYvmn5//zzT93f++PHj8fIyAjgnsW9/v7+ujs7DRs2JDIyUlefebf27dtTp04dABo0aMCxY8dK1XTeS2ZmJgCWlpYPjTk1NVWXHISEhGBsrL17HxoaSmRkJHl5eZw/f56goKAy+54/fx6AVq1a6RKNXr16sXbt2lLtXnrpJc6dO0dYWBjvv/8+vr6+tGnTptRdn4clblDxwTslA3RKvo+a5pGvyakJlAqJtq8OYVTRWyRrbMm5aUTmdVPmN7MiqI4Rd/Jh2sZz5O+ep+9QBUF4gkmShJmRQblez9Z3wMXahPuNV5UAF2sTnq3vUK7jVcXMyzY2Nrr3JbWa97qIl7fd3Uou7tnZ2RWKqaL9vFdyca/Y5s6dy/bt2xk9ejR169blwIEDvPPOO6WKosujooN3SibNrbTRyNVMJDmVpEtjF/p6NSRuhxPxv9qTdLgWSb858LFdU7rZ2XE2VcMbU+bAhR/0HaogCMJDKRUSM19qCFAm0Sn5eeZLDVEqKjd5CQoK0l3wlyxZQmFhIQAZGRkkJCRU6rkepH79+gClyinux9HRUZc8REREUFBQgCzLbNy4EdAuNtmoUaN77tu4cWMADh48qEs0IiIiyrQ7ePAgbdq0YenSpezbt0/3SO/AgQO6Nh06dMDX15cpU6bcN9agoCDdgq0l50lMTOTw4cNA2cE7Jf0vmXC3phFJTiVR/fILDh/NolZu6Vt6dgVZfGDvQA9ne94IMkLeOgKSTuknSEEQhAro0tiFFf2b4WxtUmq7s7UJK/o3o0vj+9d+/FseHh6MHj0a0I5wqlOnDv7+/tSpU4djx45V+vnup3Xr1gDlPmdJEfORI0dwd3fH09OTyMhIAKZNm3bfhVLHjx+PJEncuXMHHx8f6tevz6ZNm8q0mzx5MnZ2dnh7e9O8eXPdY6omTZro2sTExHD58mWSk+8/tN/IyIh587RPFbZu3YqXlxcNGzYkOzsbe3t7Jk+eXKr9n39qH1nePaqsJhFJTiWQ1Wquz5kJ3PtfPDIw1qUu2Y4BSMV5sKEvZIkV0wVBePR1aezCH5Pas2HYU3wSGsCGYU/xx6T2VZLglFi6dCnLli0jICCA7OxsYmNjadKkSaki4KrWs2dPDAwMiIqK0o3ufZD+/fvz/fff88wzz5CVlUVKSgoBAQF89dVX9x1ZBRAYGMj69evx8PAgPz8fd3d3VqxYUaZdnz59CAoKQqVScfbsWWxsbAgNDWXDhg0V7tvw4cMJDw8nICCApKQkJEkiJCSEQ4cOUbt2bV2748ePk5ycjJ2dHZ06darweR4FklyRyQseMyqVCmtra9301f9WVtRhEgYPeWi7Wa0GMbfOGjJTbuDs3ZQ64/aAYcXW7hIEQSiP/Px8YmNj8fT0xMTE5OE7CGX06dOHzZs3s3TpUt588019h1Ptxo8fz+LFi5k0aRILFiyo9vM/6He4vNdvcSenElyNKTsCIeLOHb78x3ofT9XO4ZVz7Wn1dQ49Pz1CwZbX4cnNMQVBEB5ps2bNQqFQsGTJklJrVz0JMjMzWbVqFZaWlkycOFHf4fxreh1C/ri4bQ5337g9n5/Puze1j6OslUp6/1XZ3yjQHkepEVd+/oaohDzGLVrP8jqN4Lma+wskCILwuPLz83vikpsS1tbWupFVNZm4k1MJLIOCSbcEzV8/NzIxYbittnp99s0UdmapSLeEWsFP8c2Yl3Dprq18X3GsiNWLZogVygVBEAShCogkpxI0c2nB9y/aIvF3ojPW3p5e1jbIwKTkZP4oLiDQIQAPe3OWTQ7DutVrAIzcns+JZUMg+bS+whcEQRCEx5JIciqBUqHkhf/NZlGIklt/TY4pSRIznJxoZ2NJkSzz/unr/DJsOLJGQ7cmtXlj/CTM6gVRoIaQdbfIWNULsh48A6cgCIIgCOUnkpxK0tG9I6EjljDn7drMek3BJy8reK+fkuJPmuDiYU6urKH/unWcnToNWZaZ/lIjngmbhZGNE9czZZbti4eNr0FRvr67IgiCIAiPBVF4XIk6uneknVs7TqSeIC03DQczB5o5NmNH8x0M6xhK7yJzDCIjSXdzxWH0aL4Y1oZO12ehvriLCW3/hMRjsO0NCPkSqmAKdEEQBEF4kog7OZVMqVAS5BxEV6+uBDkHoVQoebnRyyz65UtM+joiSRLpn37GrbXheNqbs/j1l5GfHsqwgjFoJAM4+x38/rG+uyEIgiAINZ5IcqpJP/9+eIW9waZnFeRqNEwYO5bETZt4qWlt+gXX5ZCmEe8V9ef17Xlc2jwLLv6o75AFQRCeaJcuXUKpVOLl5VVqKPnatWvx8fHB0NAQSZJYvXp1uY8pSdJD99m/f7+u3f79+/99B/6Dkkn2rKysyPjHnG81iUhyqtHIpiMxGBLKgNwkVt++Re+wMNJ37uLdbg3xc7Hio59iWHm8iO6b8sjaMBSSz+g7ZEEQBNCoIfZ3OLtF+/+aJ2PumFmzZqHRaBgzZgxKpRLQrtYdFhbGlStXcHJyIjg4GAcHBz1HWjEHDhyga9euODg46JKplStXlmpjbW1NWFgYWVlZfPTRR3qK9L8TSU41kiSJKcFTCZjaDWOlxKGcHAaE9qHwz6Ms79eM2m37obSw41K6hsFbbiGvDxUjrgRB0K8L22BJY1jTDSLCtP+/pPFjP79XamoqERERKBQK+vbtq9seHR1NUVERADt37iQqKooXX3xRX2H+KydOnGD37t3Y2to+sF2/fv0AWLVqla7PNY1IcqqZUqFk9Yg1tJvdCaUEOzMzGf7KyzglXePDgc/h8OoUJIWSrReL+fDna7CpnxhxJQiCflzYBpsHgiqp9HZVsnZ7FSU6siyzfPlyAgMDMTU1xdLSkpYtW3Lq1CkABg8ejCRJtG3bls8++wwPDw8sLS3p1q0bKSl/L37ctm1bJEliwIABzJw5ExcXF2rVqkX//v3Jysp6YAxbtmyhuLiYli1b4uTkBGjv7Ny9Gre/v3+pR0rbtm2jdevWWFhYYGpqSrNmzfj6668f2t/Nmzfj5eWFqakpXbt2JTExsUyblJQU+vXrh4uLC0ZGRjg4ONC2bVt27Niha+Ph4YEkSQwePPiB5xswYAAqlYpdu3Y9sF2LFi1wdnYmPT2dPXv2PLQfjyKR5OiBkdKI7yZGEDyuJRKwKT2dt7u+QBfLfAZ3f55anUYCMHVfAbt/Oww/jhFrXAmC8N/JMhTmlO+Vr4Kf3wHu9XfPX9t2TtK2K8/xKvB32JgxYxg9ejSnTp3C3Nwcd3d3Tp8+TVxcXKl2hw4dYuLEiRgZGZGdnc2OHTuYMGFCmeNt2rSJxYsXY2pqyp07d1i3bt1DF5z8/fffAWjZsqVum6urK35+frqfAwICCA4OxsrKivDwcF555RUOHjyIhYUFTk5OnDx5krCwMObOnXvf85w6dYq+ffsSGxuLsbEx0dHRjBgxoky7UaNGsX79erKzs/H398fU1JQDBw7w559l1058GDs7O0xNy7c4dEn/S76PmkYMIdcTCyMLfnh/Bx1uP8eZby6yIjERhy5dmLpnHyc69+JgUjQ5Z3fTNyKP43YbcHfwhWfH6ztsQRBqsqJcmFe7kg4ma+/wLHArX/OpSWBk/tBmcXFxLFu2DICQkBA2bNiAkZERaWlp5OeXvqutVqs5evQoAQEBhISEEBkZyd69e8sc08TEhIsXL+Li4kJwcDDHjh1j7969D0w+rly5AmjvjpQYOnQo3t7etGvXDoDIyEjd5927dwcgODiY3377DSMjI3r06EFkZCRz585l3LhxmJmZlTnPxx9/jEajwdramsuXL+Pk5MTAgQNZu3btPeP59NNPdXdqkpOTyczM1LWpV68eJiYmuLi4UFnc3d1Lnb+mEXdy9Mje1J4fl+yk/qt1sTZU0qSomJvDh7LsRU9cu72BkbM3RUpTErM0sHcOXNrx8IMKgiDUYH/++SfyX3d9xo8fj5GREQAODg64uZVOqPz9/QkICACgYcOGgLaW5p/at29PnTp1UCgUNGjQAICbNx9c71iSPFhaWj405tTUVOLj4wFtYmZsbIwkSYSGhgKQl5fH+fPn77lvyfZWrVrpHov16tWrTLuXXnoJgLCwMLy9venWrRvh4eHUrv130rp3714uXbrE/PnzHxpzeVlZWQGUSqZqEnEnR8/qWtUl8sufeOPr/thH5FN0PR7jd8by4dsLGHVnKhLg0fo4RK+FiGEQtguc/fUdtiAINZGhmfaOSnlcPwTrej68Xb8t4P5M+c5dyWxsbHTvDQy0lzP5Ho/FytvubiUX9+zs7ArFJFVwIteSOO7e716xzZ07l1atWrFr1y7OnTvHgQMH2LFjB/v37y9Vl1PZSlYiL/k+ahpxJ+cR0Mi+EQsGrGTBa8bcMYcjJ09SNOdNBj/XGKWVI91jXqKw7nPk5GTDhr6QXfZfKoIgCA8lSdpHRuV51WsPVrWB+120JbCqo21XnuOV8+IfFBSku+AvWbKEwsJCADIyMkhISKiEL6F86tevD8D169cf2tbR0ZG6desCEBERQUFBAbIss3HjRgBMTU1p1KjRPfdt3LgxAAcPHtTdhYqIiCjT7uDBg7Rp04alS5eyb98+3SO9AwcO6Np06NABX19fpkyZUt5uPlRJ/729vSvtmNVJJDmPiGCXYN565QMmPl3IkBvxDNm7l9abP6CRgyk3c4p58c8gPD/NI+p8LGzqL0ZcCYJQtRRK6PLBXz/8M0H56+cuC7TtKpGHhwejR48GtCOc6tSpg7+/P3Xq1OHYsWOVeq4Had26NUC5z1lS33PkyBHc3d3x9PQkMjISgGnTpt2zHge0j+QkSeLOnTv4+PhQv359Nm3aVKbd5MmTsbOzw9vbm+bNm+uKk5s0aaJrExMTw+XLl0lOTn5grFu3bsXb25u2bdvqts2YMQNvb2/dsPESJYXNd48qq0lEkvMI6ezRmaFD3sXM1QSVRsPAbT8w7tS3WBhIHPr1F9Kyi+nxXQE3Lx6GH8eKEVeCIFSthi9D72/B6h+FrFa1tdsbvlwlp126dCnLli0jICCA7OxsYmNjadKkSaki4KrWs2dPDAwMiIqKIj09/aHt+/fvz/fff88zzzxDVlYWKSkpBAQE8NVXXzFt2rT77hcYGMj69evx8PAgPz8fd3d3VqxYUaZdnz59CAoKQqVScfbsWWxsbAgNDWXDhg0V7ptKpSImJqbUXaq0tDRiYmJKDV8/fvw4ycnJ2NnZ0alTpwqf55EgP8EyMzNlQM7MzNR3KKUs+HWBbO1gLANyXUND+cf/jZLdxm6SDe1cZUB+zt1ALpxuKcu/L9J3qIIgPKLy8vLkCxcuyHl5ef/9YOpiWb52QJbPfKf9f3Xxfz9mDdC7d28ZkJcuXarvUPRi3LhxMiBPmjRJL+d/0O9wea/fkiw/ubcDVCoV1tbWujU6HhWyLPP2j2/zxcBPyc4sws/YmLf7juJdAx9S1k5AU5DLW8FGLO5iCqHrwbervkMWBOERk5+fT2xsLJ6enpiYmOg7nBrp4sWLNG7cGA8PD6Kjo3VLOzwJMjMzdaPZYmNjsbOzq/YYHvQ7XN7rt3hc9QiSJIkPu31IyKd9MTVVcrGggJWbVjIyPw67rm8BsORIIevPFkLEUEg5p9+ABUEQHkN+fn6o1WpiYmKeqAQHtGtXqVQqVCqVXhKcyiKSnEeUUqHki9e+oMOHL2BkqMBSqeSlI5H0NjPE6uneAAzdXsDFJBVsCIXsND1HLAiCIAiPFpHkPMKMlcasG76O5xa1p313V4wVCkYd38yLXk0w9QzkxR6vUa9ePci8oR1xVVyg75AFQRAE4ZEhkpxHnJWRFeuHrOf3l+uyJ0ACjYZnD3zJ88/1Ja7RILJD1oGxNdyIgh/fEiOuBEEQBOEvIsmpARzMHPj8+S/Y8rIdIwtuMurGdTx3LsLu+mXG7Mmh4NVV7L6mgdPr4dBSfYcrCIIgCI8EkeTUEO5W7ix/fiXXn7IAYMnNFIJ++pAbR0/i/9pMnl+bzfboItg9Ey7/rOdoBUEQBEH/RJJTgzSyb8SGRZtw6ewAwLzEeLr88jEKtMPn+m9Tc/WWWjvi6ua9F4MTBEEQhCeFSHJqmGdqP8Oq5atxeMYGDTArPobh2Qm41/UjM6eQkEiJnOws7YirnIfP0ikIgiAIjyuR5NRAXb268tHKT7BtYkmhLDPj2iUmmJngaG3H2YQshu1SIt++LkZcCYIg/AeXLl1CqVTi5eWFWq3WdziPhIEDByJJEl9++aW+QykXkeTUUAP9BzJ15Uxs6pmRr9FQrLrJ8np+mBsYsuHEHT45roT4w7B9nBhxJQiC8C/MmjULjUbDmDFjnrjJAO9nwoQJALz33nsUFRXpOZqHE0lODTb+qfEMX/4GzUZ60MrRioY56Sxv2BxDSWLWgSJu5wOn1sGhT/UdqiAIQo2SmppKREQECoWCvn376jucUgoLC/V27qZNm9KoUSNu3LjB9u3b9RZHeYkkpwaTJIm5HecS3Ksb83oryTcEj5x0Fvg1Y8cv+6n16gfahrtnwOWd+g1WEAShnGRZZvny5QQGBmJqaoqlpSUtW7bk1KlTAAwePBhJkmjbti2fffYZHh4eWFpa0q1bN1JSUnTHadu2LZIkMWDAAGbOnImLiwu1atWif//+ZGVlPTCGLVu2UFxcTMuWLXFyctJt37lzJ88++yyOjo4YGRlhZWXFc889x86df/8d6+vriyRJjBs3TrctNzcXc3NzJEni888/B7TrQ40dOxZ3d3eMjIxwdXVl/Pjx5Obm6va7u68ffvghrq6uunWc1q5dS8uWLbG3t8fQ0JBatWrRuXNnjh49Wqov+/fvx9/fHxMTE1q3bs2OHTuQJAlJkli9erWu3aVLl+jVqxcODg4YGxvj5+d3zxXRX3rpJQA2btz4wO/wkVAVK4fWFI/qKuQVlVeUJw/8aaD84pvech1DQznQxFRe/eoQubioWJa3jZHlmVayPLe2LKec13eogiBUkwet4JydnX3f1z/bP6htbm5uudpW1BtvvCEDMiDb2dnJjRo1ko2MjOTIyEhZlmV50KBBMiAbGhrKJiYmcv369XXtX3vtNd1x2rRpo2tnaWkpe3p66tpNnTr1gTGEhobKgDxmzJhS2xcuXCgbGhrKXl5ecmBgoGxhYSEDsoGBgXzq1ClZlmV57ty5MiDXqVNHVqvVsizL8oYNG2RANjExkW/fvi3n5+fLAQEBum1NmjSRTUxMZEBu3769rNFoSvXVyMhIVigUsp+fn+zg4CDLsiyPHj1aNjExkX18fOSmTZvKxsbGMiBbWlrKycnJsizLcnJysmxubi4Dsqmpqezr66v7GZC/+eYbWZZlOTo6Wra2tpYB2dbWVm7cuLEsSZIMyLNnzy71HWzdulUGZBcXlwr/t62IyliFXCQ5j0GSI8uyfCf/jtz+k/aykYlCBuRnzc3l7SMmygf27ZWndfPWJjqLG8tydpq+QxUEoRo86AJRcoG716tr166l2pqZmd23bZs2bUq1tbe3v2e7ioiNjdVdXENCQuSCggJZlmU5NTVVjo+Pl2X57wu/QqGQT548KcuyLHfv3l0GZCcnJ92xSpIcS0tLOSEhQVar1XKLFi1kQA4ODn5gHM2bN5cBedGiRWXiu337tu7nW7duyZaWljIgT58+XZZlWY6Pj5cVCu3fxb/99pssy7L86quvyoDcp08fWZZlefXq1brkJTo6WpZlWT516pTuO9uzZ0+pvgLyjz/+KMuyLBcXF8uyLMuXLl2Sc3JydLFcuXJF13bVqlWyLMvy9OnTdd/V6dOnS227O8kZPHiwDMiNGzfWHXPJkiW65EilUunOc/z4cd3+/yaJLa/KSHLE46rHhLWxNevD1tN0enMMDCR+z8lh5frPWdc7lLnbr/JNtDXciYdNA6BYf89zBUEQHuTPP/9E/muwxPjx4zEyMgLAwcEBNze3Um39/f0JCAgAoGHDhoC2luaf2rdvT506dVAoFDRo0ACAmzdvPjCOzMxMACwtLUttLywsZPDgwTg6OqJUKrG1tdU9+kpKSgLAzc2Ndu3aAdpHOiqVip9/1k7SOmjQIADdI6XCwkJ8fHyQJEnXF4CoqKhS5/Xx8aFbt24AuiLozMxMXnnlFWxtbVEoFNSvX1/XviSW8+e1c6b5+vrSpEkTAHr37l2mvyXxnDt3TvdY7a233gIgLy+PM2fO6NpaWVmV+Z4eVQb6DkCoPE7mTmx8YyMvqV7i0kcX2J6lor+yFr2trXl9602aDLWhOYdgxzh4+TOQJH2HLAiCHmRnZ9/3s3+OIrpX0lBCoSj97+S4uLj/FFdF2djY6N4bGGgvZ/I9RpOWt93dSi7k//yuunXrxpUrVzAwMNDVuZw8eZLCwsJSw8wHDRrE3r17iYiIoEWLFhQUFODi4sLzzz9f6vxGRkYEBgaWOX+tWrVK/ezs7Fzq5+zsbDp37sydO3cwMTEhMDAQQ0NDjhw5AlBmyLt019/39+p7yTZ7e3vtws//cPfvhUql0r2/O+F5FIk7OY8ZL2svNr69EffhngCE37mNk6EhHYyMCYmE9FzgZDgcXqbfQAVB0Btzc/P7vkqKWsvT1tTUtFxtKyIoKEh3QV6yZIluJFFGRgYJCQn/odcVU3JX5Pr167ptGRkZXLlyBYA5c+Zw6tQpNm7cWCqBKNGjRw8sLCxITU1l2rRpAPTv31+XLLRs2RLQJiPLly8nKiqKqKgo9u/fz8SJE3nttddKHe+f57h8+TJ37twB4Ouvv+b48eMsWbKkTByNGzcGtEXFFy9eBOC7774r064kHmtra3766SddPNu3b2fcuHE89dRTurYl34mTkxMWFhZljvUoEUnOY8jfwZ+109dSu19tAHaoVMx0csYjM4fQX10o1siw+12I/kXPkQqCIJTm4eHB6NGjAe0Ipzp16uDv70+dOnU4duxYtcXRunVrgFLntLW1xdXVFYCZM2fi7+9Ps2bN7jmHjpmZGT179gTQjfgaPHiw7vO+ffvSpEkT1Go1QUFBNG7cmAYNGmBjY0PPnj11Ccz9eHl56RLIsLAwmjRpwquvvlqm3ahRozA3N9edx8/Pj0WLFpVpN2XKFKysrIiJicHNzY3AwEDc3d1xdnZm0qRJpdqWPNp69tlnHxjjo0AkOY+pVnVa8fmcz6kzuDajO7ljqVSypHYd7pyPZ/r5+iBrYMsQSL2o71AFQRBKWbp0KcuWLSMgIIDs7GxiY2Np0qQJHh4e1RZDz549MTAwICoqivR07RI5kiTpHj8plUrUajXr1q3DwcHhnscoqb8BaNGiha5uCMDY2JjffvuNMWPG4ObmRnR0NLdv36ZFixbMnTu31LD1e6lVqxbfffcdDRs2RKPRYGRkxI8//limnbOzM9u3b6dRo0YUFRVhY2PDqlWrdJ+X3I1r0KABhw8fplevXpiZmXH+/Hk0Gg1dunThvffeK3XMkvlxQkNDHxjjo0CSH/Zg8jGmUqmwtrYmMzPzkX+u+G99e/5bFh35kLe3amh+VeZGYSE7Gvmx+FUNihsHwcYdhv0K5nb6DlUQhEqUn59PbGwsnp6eZR5BCeXTp08fNm/ezNKlS3nzzTf1Hc6/duXKlVJFyXPnzmX69OmA9jFWSTF2eZw5c4amTZvi5ubG1atXdYXhVeFBv8PlvX6LOzmPuYGNBtK30f9Y9KqCxdIdelyP49m4eIqemQe1PODOddgsRlwJgiD806xZs1AoFCxZsqRGr13Vq1cvGjZsyCuvvEJQUJAuwRkwYECFEhyAjz76CIDp06dXaYJTWUSS8wR4p+V4nnZ7gU1SFtkaDcMuXmDfoBEUtv+MfIUlXD8IO8aLNa4EQRDu4ufnh1qtJiYmpkavXfXCCy9QUFDAL7/8wvnz52natCkff/wxX3/9dYWP9e233yLLMsOHD6+CSCufeFz1mD+uKlGkKaL7uiHsfzeCnOt5OBsY8Gk9b76vbcO3z15EIcnQeR48PVrfoQqCUAnE4yqhphOPq4RyM1QYsjn0cxpObIuZkxEpxcVMir1G+yuJfJzUStvol+lwZbd+AxUEQRCESiKSnCeImaEZkQPW4j6hJSY2hlwrLGRJehpWP59hd/Gzd424uqTvUAVBEAThPxNJzhOmjpUdK3p8hev4xhiaKTmbn8/N4mLi1l0g1jQAClSwoQ/kZOg7VEEQBEH4T0SS8wRq4+XD6OeW4jauAf6dnXjZ1prWxsbsWZNGrqkr3I6DzQPFiCtBEAShRhNJzhNq7HOteMZ3Jpo+tVnUQ0mxJBOs1hD5ozWyoSVc/wN+eluMuBIEQRBqLJHkPKEkSWJ5z+5YqYZwop4BizvDiIQb/HEpmvSinoAEJ9bAkZX6DlUQBEEQ/hWR5DzBLE0M+aLnAIpTerGnKJeo3FxWZmSweMUGbqFdc4VdU+HKHv0GKgjCE8XDwwNJkpg1axYA+/fvR5IkJEmq9pXO/6tLly6hVCrx8vIqNaHg2rVr8fHxwdDQEEmSWL16dbmPWfJdPGifu7+z/fv3//sO/Aclw7utrKzIyNBPnadIcp5wjetYM63tAIzd++IY4gjA/NRUvvz8J66rWv814up/kHZZz5EKgqAvao2aP1P+5KdrP/Fnyp+oNdU7+6+VlRXBwcEEBwdjbGxcref+r2bNmoVGo2HMmDG6CQVTU1MJCwvjypUrODk5ERwcfN/1rx5l69evp1mzZpiammJra0vPnj11q7SDdkXzsLAwsrKydDMlVze9JzkP+5LuJTU1lddffx1PT09MTU2pVasWLVq04PPPP6+mqB8v/YPr8nydPlg/E4JdJ1sApqck89PmM9zObaIdcbW+D+Te0nOkgiBUtz3X99A5ojNDdg1h0u+TGLJrCJ0jOrPnevXd4W3WrBlRUVFERUXh4uJSbectj8LC+w/QSE1NJSIiAoVCQd++fXXbo6OjKSoqAmDnzp1ERUXx4osvVnmslemLL76gX79+nDx5EhcXF9RqNREREbRq1YqkpCRdu379+gGwatUqXZ+rk16TnPJ+Sf/Uu3dvVq5cSXx8PD4+PhgbG3P8+HFGjhzJd999V409eDxIksT8Hv44a3pi17Ur1k9bUwyMT0pk+6Yb5GS7wu1YMeJKEJ4we67vYfz+8dzMvVlqe2puKuP3j6+2ROdej6sGDx6MJEm0bduWzz77DA8PDywtLenWrRspKSml9g8PDycoKAgzMzMsLS3p0qULp06d0n2ek5PDq6++iqenJ+bm5hgbG1O/fn1mzJhRKolp27YtkiQxYMAAJk6ciKOj4wPXftqyZQvFxcW0bNlSt6r4rFmzePbZZ3Vt/P39Sz1S2rZtG61bt8bCwgJTU1OaNWtWruUXNm/ejJeXF6ampnTt2pXExMQybVJSUujXrx8uLi4YGRnh4OBA27Zt2bFjh65NyaPCwYMH3/dcBQUFTJ06FYAePXpw7do1Ll68iKWlJWlpacyfP1/XtkWLFjg7O5Oens6ePdVf+qC3JKciX9LdZFnm0KFDAAwdOpTTp09z8uRJ3efXr19/4DlVKlWpl6BlZWLI8tdaIKf1wTm0PZZNLMiXZaYkJxG9UyJPZQVxv8PPE8WIK0GooWRZJrcot1yvrIIs5h+dj0zZP+/yX/9bcHQBWQVZ5TpeVa0gdOjQISZOnIiRkRHZ2dns2LGDCRMm6D7/8MMPGTBgAMeOHcPNzQ0rKyt27dpF69atuXjxIgB5eXn88MMP5OXl4ePjg6OjI1evXuW9995j2rRpZc65efNmPvnkE5ycnLCxsblvbL///jsALVu21G1zdXXFz89P93NAQADBwcFYWVkRHh7OK6+8wsGDB7GwsMDJyYmTJ08SFhbG3Llz73ueU6dO0bdvX2JjYzE2NiY6OpoRI0aUaTdq1CjWr19PdnY2/v7+mJqacuDAAf7888/7f8H3cOzYMV2NTY8ePQCoXbs2Tz31FAC7du0q1b6k/yXfR3UyqPYz/uVBX9Lu3bvLfEklJEmiVatW7N+/n1WrVnHkyBFSUlKQJImuXbsybNiw+55z/vz5zJ49u/I785hoXMea6d0aM2PbAOqE5ZD8bRRvKmthkqkmdr899drkYXx8NTj4wVMj9R2uIAgVlFecR/D64Eo73s3cmzyz8ZlytT3y2hHMDM0q7dwl1Go1R48eJSAggJCQECIjI9m7dy8Aubm5ur/zZ8+ezYwZMyguLubpp5/m2LFjzJs3j7Vr12JlZcX58+dp2LCh7rgDBgwgPDycjRs3snDhwlLnlGWZI0eOEBgY+MDVyUtKLzw8PHTbhg4dire3N+3atQMgMjJS93n37t0BCA4O5rfffsPIyIgePXoQGRnJ3LlzGTduHGZmZb/Djz/+GI1Gg7W1NZcvX8bJyYmBAweydu3ae8bz6aef6u7UJCcnk5mZqWtTr149TExMHvhY8MaNG7r3jo6Ouvcld6vi4+NLtXd3dy91/uqktzs5Ff2S7hYZGUnnzp3RaDScPn2amzdvYm5uTvPmzbG0tLzvflOmTCEzM1P3ujsGQWvAU+50bVyXvJthuI9swXdDLbhsLyPl5hN70I3CbCXsmgJXxYgrQRD0z9/fn4CAAABdkpKamgrA+fPnyc3NBWDmzJlIkoShoSHHjh0DICoqCgClUkl4eLiu/EGSJMLDwwHuWTrRrl07AgMDdfveT0ny8KDrUonU1FTddS8kJEQXR2hoKKC923T+/Pl77luyvVWrVrpraK9evcq0e+mllwAICwvD29ubbt26ER4eTu3atXVt9u7dy6VLl+77NAW47125ku2SJJXaXrKA5t3JVHXR252cin5Jd5syZQq7du2iZ8+efPXVV5w5c4YOHTowZ84catWqxVtvvXXP/YyNjWtcZX51kySJBT2acC5RxY24IdSqt4IF/e/Q45NMtpyLZ6XkjX/beAy+GwJD94CDj75DFgShnEwNTDny2pFytT1+8zij9o56aLvlHZbT3Kl5uc5dFe5+XGRgoL2klVxH7r7O+Pn5lVmt2s7ODoAFCxboLuru7u44OzuTkJBAYmIiGo2mzDmdnZ3LFVvJ+bKzs8vZG60HXf/u5V7XzXtdY+fOnUurVq3YtWsX586d48CBA+zYsYP9+/eXqst5mLp16+re37z5d71WSXLp5uZWqn1JaciDVguvKnq7k1PRL6nElStXWLlSO0Hda6+9hpWVFa1bt8bX1xdAL4VNjxsrE0OWvdYMQ9mWO7FDyDcw5t3UmxzNy2X42StE/14HddZfa1yJEVeCUGNIkoSZoVm5Xs/UfgYnMyck7n3BlZBwNnPmmdrPlOt4Fb1wV4bGjRtjaqpNrrp06cLhw4d1o7RWrFihq7cpuaPj4+NDXFwchw4domnTpvc9bnn7Ur9+feDBtaIlHB0dddfFiIgICgoKkGWZjRs3AmBqakqjRo3u20+AgwcP6q6hERERZdodPHiQNm3asHTpUvbt28eyZcsAOHDggK5Nhw4d8PX1ZcqUKfeNNSgoSJcglpwnMTGRw4cPA9rv+m4l/ff29n7QV1Al9JbklPdL8vX1xdfXl88++wwofbur5JZjRkaGruLe3Ny8WuJ/3Pm7WjO1qy+aQidyk8PwGFcPpYmCY3l5vHH2OrF/OKNJjYXvBoG6+ocFCoJQtZQKJZNbTgYok+iU/Dyp5SSUivs/rtE3MzMz3n33XQAWL16Mq6srAQEB2NnZ0axZM3755RcAmjRpAmiHdnt6elK3bl1d4vNftG7dGvj7WvUwJcXFR44cwd3dHU9PTyIjIwGYNm3aPetxAMaPH48kSdy5cwcfHx/q16/Ppk2byrSbPHkydnZ2eHt707x5c11xckn/AWJiYrh8+TLJycn3jdPIyIh58+YBsHXrVry8vGjYsCHZ2dnY29szefLkUu1LCpvvHlVWXfSW5JT3S7p8+TKXL18mPT0dgKZNm1KvXj0A5s2bR8OGDalfv77udtjAgQP10JvH06BnPOjSyJnCHHesLV7HY5wnCgOJX7OzmXg6hRsH7ZGvHoCf3xEjrgThMdTRvSOL2i7C0cyx1HYnMycWtV1ER/eOeoqs/KZMmcKaNWsICgri9u3bXL16FUdHR0aOHElISAgAU6dOZeDAgdjY2KBSqQgNDWXUqIc/qnuYnj17YmBgQFRUlO4a9iD9+/fn+++/55lnniErK4uUlBQCAgL46quv7jnKq0RgYCDr16/Hw8OD/Px83N3dWbFiRZl2ffr0ISgoCJVKxdmzZ7GxsSE0NJQNGzZUuG/Dhw8nPDycgIAAkpKSkCSJkJAQDh06VKrG5/jx4yQnJ2NnZ0enTp0qfJ7/TNaz8PBwOSAgQDY2Npatra3lkJAQOTo6Wvc5IAPyzJkzddtu3Lghjxw5Uvb09JRNTExkBwcHuW3btvJPP/1UoXNnZmbKgJyZmVlZ3Xns3MktlFt/sFd2n7RdfnXNYrnu2LqypND+N/lfLVv5Rkd3WfOulSxHfa7vUAVBuEteXp584cIFOS8v7z8fq1hdLB9NPirviNkhH00+KheriyshwidD7969ZUBeunSpvkPRi3HjxsmAPGnSpArv+6Df4fJevyVZfnL/Ca5SqbC2ttatryHc2+kbd+i58hBFapl2gUfZvfsrEr/STjS10KU2/QKUOLfMRhqwBeq113O0giAA5OfnExsbi6enJyYmJvoO54l18eJFGjdujIeHB9HR0Q8cjfW4yczM1NXXxsbG6kpUyutBv8PlvX7rfVkH4dHX1M2GqV21k1cdPBNMxw4v49zHGQ8fSzpYWnDnmjlpp81g82BIr/55EARBEB5Vfn5+qNVqYmJinqgEB7RrV5VMvFvRBKeyiCRHKJfBz3jQuZEThWqZmBs9aNk+GPPJdfn6ee0f2oyLlqSfUos1rgRBEIRHhkhyhHKRJIkPezTFtZYpN27l42w6GavbVuxvoWRNK/g4LZUdh5XcPposRlwJgiAIjwSR5AjlZm1myGevNcNQKbHrfAb9mi6DFPi2MJOvbt1idGIi+/4wRvXbUfh5kr7DFQRBEJ5wIskRKiTAzYbJL2jrcz7Zl8LMZ5bT8PmGmPuak6PRMDwhgYO/mZP9Yzgc/VLP0QqCIAhPMpHkCBU2pJUHnRo6UajW8GlUMcu6fkWzSc0wcTfhllrN0PgbHP/Vgtxvp0PMPn2HKwiCIDyhRJIjVJgkSXzUsyl1bEy5npHL0p0ZfPnylzR4uwFGTkYkFRcz9HoiZ/dakv/5EEi/qu+QBUEQhCeQSHKEf0VbnxOIgUJix9lkjkab4HLahbpv1cXQxoCrhYWMup5E3C5DClf0grzb+g5ZEARBeMKIJEf41wLr1mLyC9qFUd/78QIzxn+O8qCSuuPdMbQyIMTbFbnAgPjIbIq+7idGXAmCUC4eHh5IksSsWbMA2L9/P5IkIUmSbp3CmuLSpUsolUq8vLxQq9W67WvXrsXHxwdDQ0MkSWL16tXlPmbJd/Ggfe7+zvbv3//vO/AflEzUZ2VlRUZGhl5iEEmO8J+Etfako58jhWoNU7bHsHbOJrJ/z6b+Qh9+HmlEvpMVRbkGxK+JpnjLBH2HKwjCvyCr1eQcOUrm9h3kHDmKfNfFujpYWVkRHBxMcHAwxsbG1Xru/2rWrFloNBrGjBmjmwwwNTWVsLAwrly5gpOTE8HBwTg4OOg50oo5cOAAXbt2xcHBQZdMrVy5slQba2trwsLCyMrK4qOPPtJLnCLJEf4TSZL4qJe2PicuI5c1F9V8OuxT0rankWkuMb5HDglmBiyPVRG/ZDvqA8v1HbIgCBWg+uUXrnboSPygQSS9/TbxgwZxtUNHVH+t4F0dmjVrRlRUFFFRUbi4uFTbecujsLDwvp+lpqYSERGBQqGgb9++uu3R0dEUFWnvbO/cuZOoqChefPHFKo+1Mp04cYLdu3dja2v7wHb9+vUDYNWqVbo+VyeR5Aj/mY2ZEUv7autztp9JpsCtJYN8B5GxJ4NUE5lX4i6zKD2NJVeySJj+MZpLu/UdsiAI5aD65RcSx75FcUpKqe3FN2+SOPatakt07vW4avDgwUiSRNu2bfnss8/w8PDA0tKSbt26kfKPeMPDwwkKCsLMzAxLS0u6dOnCqVOndJ/n5OTw6quv4unpibm5OcbGxtSvX58ZM2aUSmLatm2LJEkMGDCAiRMn4ujoSIMGDe4b95YtWyguLqZly5Y4OTkB2js7zz77rK6Nv79/qUdK27Zto3Xr1lhYWGBqakqzZs34+uuvH/odbd68GS8vL0xNTenatSuJiYll2qSkpNCvXz9cXFwwMjLCwcGBtm3bsmPHDl2bkkeFgwcPfuD5BgwYgEqlYteuXQ9s16JFC5ydnUlPT2fPnj0P7UdlE0mOUCmau9diUhdtfc6c7Rfo/8ZkfJN8scu0w6arPQCfZqTz9eUcEt8chXwzWp/hCsITSZZlNLm55Xqps7K4+f5cuNcazrIMyNycOw91Vla5jldVa0EfOnSIiRMnYmRkRHZ2Njt27GDChL8fjX/44YcMGDCAY8eO4ebmhpWVFbt27aJ169ZcvHgRgLy8PH744Qfy8vLw8fHB0dGRq1ev8t577zFt2rQy59y8eTOffPIJTk5O2NjY3De233//HYCWLVvqtrm6uuLn56f7OSAggODgYKysrAgPD+eVV17h4MGDWFhY4OTkxMmTJwkLC2Pu3Ln3Pc+pU6fo27cvsbGxGBsbEx0dzYgRI8q0GzVqFOvXryc7Oxt/f39MTU05cOAAf/755/2/4Puws7PD1NS0XG1L+l/yfVQng2o/o/DYGvqsJ1HXMth7KZW3Np9lc8QPWFkYMXrfaLZlbyPthzTeS72J9XkloSNCcFn3G5K5fhZtE4QnkZyXx+VmzSvpYNo7OtFBLR/eFmhw4jiSmVnlnPsuarWao0ePEhAQQEhICJGRkezduxeA3NxcZs+eDcDs2bOZMWMGxcXFPP300xw7dox58+axdu1arKysOH/+PA0bNtQdd8CAAYSHh7Nx40YWLlxY6pyyLHPkyBECAwNLFRP/05Ur2gWLPTw8dNuGDh2Kt7c37dq1AyAyMlL3effu3QEIDg7mt99+w8jIiB49ehAZGcncuXMZN24cZvf4Dj/++GM0Gg3W1tZcvnwZJycnBg4cyNq1a+8Zz6effqq7U5OcnExmZqauTb169TAxManUx4Lu7u6lzl+dxJ0codKU1OfUtjYhNj2H936+ipHSiE/afULbIW2xbW+LDExKTuKn4ypujuyGXCxGXAmC8O/5+/sTEBAAoEtSUlNTATh//jy5ubkAzJw5E0mSMDQ05NixYwBERUUBoFQqCQ8Px8fHB2NjYyRJIjw8HICkpKQy52zXrh2BgYG6fe+nJHmwtLR8aD9SU1OJj48HICQkRBdHaGgooL3bdP78+XvuW7K9VatWusdivXr1KtPupZdeAiAsLAxvb2+6detGeHg4tWvX1rXZu3cvly5dYv78+Q+NubysrKwASiVT1UXcyREqVS1zIz59LZDen0ex7XQST3nZ0TPQGdVqFbadbFFnq8k8msmYxASsDiho93ZPHJb8oO+wBeGJIJma0uDE8XK1zT12jBvDyz7y+Ce3Lz7HrEWLcp27Ktz9uMjAQHtJK3k0dvcjMj8/P93FtoSdnfZO8oIFC3QXdXd3d5ydnUlISCAxMRGNRlPmnM7OzuWKreR82dnZ5eyNliRJFWpf0s+797vX48G5c+fSqlUrdu3axblz5zhw4AA7duxg//79pepyKptKpQIo8/1XB3EnR6h0zd1teaezthhv1o/nuZqej7uDO9cXXccl1AWLxhbY2JnibGhI+s5obs0dreeIBeHJIEkSCjOzcr3MW7XCwNkZ7nfBlSQMnJ0xb9WqXMer6IW7MjRu3FhXN9KlSxcOHz6sG6W1YsUKXb1NyR0dHx8f4uLiOHToEE2bNr3vccvbl/r16wNw/fr1h7Z1dHSkbt26AERERFBQUIAsy2zcuBEAU1NTGjVqdN9+Ahw8eFB3FysiIqJMu4MHD9KmTRuWLl3Kvn37WLZsGaAdDl6iQ4cO+Pr6MmXKlHL1sTxK+u/t7V1pxywvkeQIVWLYs160a+BAYbGGN9afYN7CRQR6BhK3JA63YW7YzHYn7nntvBA31+4jc9WHeo5YEIS7SUolTlP/utD986L+189OU6cgPeBxjb6ZmZnx7rvvArB48WJcXV0JCAjAzs6OZs2a8ctfo8OaNGkCaId2e3p6UrduXV3i81+0bt0aQPd47GFKiouPHDmCu7s7np6eREZGAjBt2rR71uMAjB8/HkmSuHPnDj4+PtSvX59NmzaVaTd58mTs7Ozw9vamefPmuuLkkv4DxMTEcPnyZZKTkx8Y69atW/H29qZt27a6bTNmzMDb21s3bLxESWHz3aPKqotIcoQqoVBIfNw7ABdrE66l5zDnp2i2bNmCZZ4l8SviMTAxYGFLFdEtzdmTlcW5D74g64f1+g5bEIS7WD3/PHU+WYLBX3UeJQycnKjzyRKsnn9eT5GV35QpU1izZg1BQUHcvn2bq1ev4ujoyMiRIwkJCQFg6tSpDBw4EBsbG1QqFaGhoYwaNeo/n7tnz54YGBgQFRVFenr6Q9v379+f77//nmeeeYasrCxSUlIICAjgq6++uucorxKBgYGsX78eDw8P8vPzcXd3Z8WKFWXa9enTh6CgIFQqFWfPnsXGxobQ0FA2bNhQ4b6pVCpiYmJK3aVKS0sjJiam1PD148ePk5ycjJ2dHZ06darwef4rSa6qcX01gEqlwtraWjf1tFD5jsXdos8XUag1MgtC/HHKiaFTp06YB5hT94263D5wi8RvkmhsYsJqdzd8v1yJ+bPt9R22INR4+fn5xMbG4unpiYmJyX86lqxWk3vsOMVpaRg4OGDWovkjfQfnUdKnTx82b97M0qVLefPNN/UdTrUbP348ixcvZtKkSSxYsKBC+z7od7i8129xJ0eoUi08bHn7eW19zsxt53H2bc6HH36I6riKjE0ZmNU3R2mh5Fx+PqNvJHLt9dHknT6t56gFQbibpFRiHtwS624vYh7cUiQ4FTBr1iwUCgVLlix54HDzx1FmZiarVq3C0tKSiRMn6iUGMbpKqHIjnvPiSGwG+y+nMXr9CbaNfpPExET69evHEeURPsr/iLgP4jiSm8vb1xNZ8r8BeH0XiXG9evoOXRAE4T/x8/N74pKbEtbW1rqRVfoi7uQIVU6hkFjUOwBnKxOupeXw7g/n+eijj2jWrBkjm4zkfy/8j7pj6iIZSPySncXMaze43j+UontMSy4IgiAI5SWSHKFa2P41f45SIRF5MpHvjiUA2qGYnRSd8PX0xXWEK0jwXeYdFkXHEj/wNYozMvQcuSAIglBTiSRHqDZBHraM7+QDwLs/nONSioqkpCTatmnLrjG78A/yp/Yg7cybakMNBQk3iR88EHVWlj7DFgRBEGookeQI1er1NvV4zseBgmINo9edwNrOkXfeeQe5WObXt34loEsAXu96kf+OKwpTDQVXrnFj+DA0eXn6Dl0QBEGoYUSSI1QrhUJice+mOFkZE5OWw7vfn2PGjBl07dqVvDt5XHz/It7+3pyyk/msjyEFymKiDkWRMGYMcmGhvsMXBEEQahCR5AjVzs7CmKWhgSgk2HoykYiTSYSHh+Pl5UXsuVjUm9TYmtiyr5bMKzmJDL4Rz4Fdv5A0eQryEzpKQRAEQag4keQIehHsZceEv+bPmfHDOdIKDYiMjMTU1JS9EXtpEt0ECzMLbtsYUCDLjEpM4M+tW0l5//17LjwnCIIgCP8kkhxBb15vU49n69uTX6Rh1LoTePs25MsvvwQg/lg8n3T4BK83vDDzMSNLo2FYwg3OrvmWtCWf6DlyQRCqkoeHB5IkMWvWLAD279+PJElIkkRcXJxeY6uoS5cuoVQq8fLyemLny/mngQMHIkmS7u/7qiSSHEFvFAqJxX0CcLQ05mpqNu9+f55+/fqxc+dOIiMjebrO03zQ8QM8xnpg7GpMulrN0IQbXFq2jIyvvtZ3+IIgVBMrKyuCg4MJDg7G2NhY3+FUyKxZs9BoNIwZMwalmCkagAkTJgDw3nvvUVRUVKXnEkmOoFf2FsYs7autz4k4kcB3x27QuXNnFArtr+bz7s8zpc0UPN72wNDBkBtFRQxPuMHVBQu4s2WLnqMXBKE6NGvWjKioKKKionBxcdF3OKUUPmBARGpqKhERESgUCvr27VuNUT3cg+Kuak2bNqVRo0bcuHGD7du3V+m5RJIj6N1TXnaM66idP2fGD+e5clM7L05ubi4DBw5k2+xtvPHcG3hM9MDAyoBkTRGJRUUkz5iJatcv+gxdEGqknJyc+77y8/PL3TbvH1M73K/df3Wvx1WDBw9GkiTatm3LZ599hoeHB5aWlnTr1o2UlJRS+4eHhxMUFISZmRmWlpZ06dKFU6dOlYr71VdfxdPTE3Nzc4yNjalfvz4zZswolQy0bdsWSZIYMGAAEydOxNHRkQYNGtw37i1btlBcXEzLli1xumsl9507d/Lss8/i6OiIkZERVlZWPPfcc+zcuVPXxtfXF0mSGDdunG5bbm4u5ubmSJLE559/DmjXhxo7dizu7u4YGRnh6urK+PHjyc3N1e1393f14Ycf4urqqlvwcu3atbRs2RJ7e3sMDQ2pVasWnTt35ujRo2X+G/j7+2NiYkLr1q3ZsWOH7r/J6tWrde0uXbpEr169cHBwwNjYGD8/v3uuiP7SSy8BsHHjxvt+f5VCfoJlZmbKgJyZmanvUJ54xWqN3H9VlOw+abvc8eP9ck5Bkfznn3/KhoaGMiB/uPBDedahWXK9OfVk3/e85NPdPOQLDXzli4395aw//tB3+ILwyMnLy5MvXLgg5+XllfkMuO+ra9eupdqamZndt22bNm1KtbW3t79nu4pyd3eXAXnmzJmyLMvyr7/+qjtWbGysLMuyPGjQIBmQDQ0NZRMTE7l+/fq6Nq+99pruWB988IFuu4+Pj1y7dm0ZkM3NzeULFy7IsizLaWlpMiA7OTnJAQEBsqurq26ft99+W3esNm3ayIBsZGQkGxoayo0bN5YDAgLu24/Q0FAZkMeMGVNq+8KFC2VDQ0PZy8tLDgwMlC0sLGRANjAwkE+dOiXLsizPnTtXBuQ6derIarValmVZ3rBhgwzIJiYm8u3bt+X8/Hw5ICBAt61JkyayiYmJDMjt27eXNRpNqe/KyMhIVigUsp+fn+zg4CDLsiyPHj1aNjExkX18fOSmTZvKxsbGMiBbWlrKycnJsizLcnJysmxubi4Dsqmpqezr66v7GZC/+eYbWZZlOTo6Wra2tpYB2dbWVm7cuLEsSZIMyLNnzy71HWzdulUGZBcXl/t+fw/6HS7v9VvcyREeCcq/1rdysDTmSmo2M384T4sWLfjkE22R8eRJk2mV24purbth4GbGyFdNwT2f+Jwc4t94k7y7/lUmCMKTQ61Wc/jwYaKjo+nevTsAe/fuBbR3PmbPng3A7NmzuXz5MtevX6dFixbk5OQwb948QFvzc/78eVJSUjh58iQ3btygf//+wL3vNMiyzJEjRzh79izHjh27b2xXrlwBtIXUd+vZsyepqanExMRw4sQJ4uPjsbS0pLi4mC1/PYYfMGAACoWCxMRE/vjjDwA2bdoEwCuvvIKNjQ0bN27k1KlTGBkZcebMGU6fPk1UVBQA+/btY9++faXOW1hYyA8//MCFCxdITk4G4M033yQjI4PLly9z6tQpzp07B0BWVhY7duwAYNmyZeTk5KBQKIiKiuLixYul7jCVmDdvHpmZmTRu3JgbN25w9uxZFi9eDMCCBQvIumv2end3dwCSk5Mr5W7f/YgkR3hkOFga80loAAoJvjueQMTxBEaOHMngwYPRaDT0De3LGx5v0NypOVkGSvo1VhMSH8vC69e5PnwE+Zej9d0FQagRsrOz7/uKiIgo1TY1NfW+bX/++edSbePi4u7Zrir5+/sTEBAAQMOGDXUxA5w/f1732GbmzJlIkoShoaEuMSlJCJRKJeHh4fj4+GBsbIwkSYSHhwOQlJRU5pzt2rUjMDBQt+/9ZGZmAmBpaVlqe2FhIYMHD8bR0RGlUomtra0uASg5n5ubG+3atQO0iZZKpdJ934MGDQLQPVIqLCzEx8cHSZJ038Xd/Svh4+NDt27dSsWdmZnJK6+8gq2tLQqFgvr16+val8Ry/vx5QPsIrUmTJgD07t27TH9L4jl37pzusdpbb70FQF5eHmfOnNG1tbKyKvM9VQWDKjuyIPwLz9SzZ2wHHxbviWb69+do6mbN8uXLOXPmDCdOnKBv7778tPcnRu4fyZGsI+SoZVbfvoVtbCwjhobhsX49Rm5u+u6GIDzSzM3N9d62stjY2OjeGxhoL2nyX3NpyXfNqeXn51fqwgpgZ2cHaO8yzJ8/H9DeYXB2diYhIYHExEQ0Gk2Zczo7O5crtpLz/TPR69atG1euXMHAwEBX53Ly5EkKCwtLDTMfNGgQe/fuJSIighYtWlBQUICLiwvPP/98qf4ZGRnpkq671apV64FxZ2dn07lzZ+7cuYOJiQmBgYEYGhpy5MgRgDJD3iVJ0r2X7zFfWck2e3t76tWrV+bzuxNClUqle//P/y6VSdzJER45b7T3ppW3HXlFakatOwFKIyIiIrC1teXYsWNMe3saKzuupPHzDXEO1f6hXZSexqYrV4n/3xCKbqbquQeCIDwKGjdujKmpKQBdunTh8OHDulFaK1asYNq0acDfdzx8fHyIi4vj0KFDNG3a9L7Hvfti/yAld0WuX7+u25aRkaF7jDVnzhxOnTrFxo0b73nMHj16YGFhQWpqqi7W/v3765KFli1bAtpkZPny5bq+7d+/n4kTJ/Laa689MO7Lly9z584dAL7++muOHz/OkiVLysTRuHFjQFtUfPHiRQC+++67Mu1K4rG2tuann37SxbN9+3bGjRvHU089pWtb8p04OTlhYWFR5liVRSQ5wiNHqZBY0icQB0tjom9mM2vbeTw8PNi4cSNOTk6EhobiYObA589/ic/L9bB/0R6AWTdT+OniRW4MHYr6rz+4giA8uczMzHj33XcBWLx4Ma6urgQEBGBnZ0ezZs345Rft6MySRzDR0dF4enpSt27dMo96/o3WrVsDlKrbsbW1xdXVFdA+QvP396dZs2b3fOxlZmZGz549AXQjxgYPHqz7vG/fvjRp0gS1Wk1QUBCNGzemQYMG2NjY0LNnT10Ccz9eXl66u29hYWE0adKEV199tUy7UaNGYW5urjuPn58fixYtKtNuypQpWFlZERMTg5ubG4GBgbo7Y5MmTSrVtuTR1rPPPvvAGP8rkeQIjyQHS2M+6ROAJMGmYzeIPJlAp06diImJoX379gC4W7mzvNPnePZxp9ZztdAAbycnceD0KeJHjEBThcVsgiDUDFOmTGHNmjUEBQVx+/Ztrl69iqOjIyNHjiQkJASAqVOnMnDgQGxsbFCpVISGhjJq1Kj/fO6ePXtiYGBAVFQU6enpgPZuSsnjJ6VSiVqtZt26dTg4ONzzGCX1NwAtWrTQ1R0BGBsb89tvvzFmzBjc3NyIjo7m9u3btGjRgrlz55Yatn4vtWrV4rvvvqNhw4ZoNBqMjIz48ccfy7RzdnZm+/btNGrUiKKiImxsbFi1apXu85K7ZQ0aNODw4cP06tULMzMzzp8/j0ajoUuXLrz33nuljlkyP05oaOgDY/yvJPleD9aeECqVCmtrazIzM6v0maDw7y3ZE82SPVcwM1Ky7Y3WeDv+fVszOjoaa2trYtQxjNr1OrHL4lAdV9HZxprFTi6YP/M0ritXojAy0mMPBEE/8vPziY2NxdPTUzcnilD9+vTpw+bNm1m6dClvvvmmvsP5165cuVKqKHnu3LlMnz4d0D7GetB8Qf905swZmjZtipubG1evXsXoPn9HP+h3uLzXb3EnR3ikvdm+Ps/UsyO3UM0b60+QX6QthPvll18ICgoiNDSUlo4tmd92Aa4jXHHq6cRrw2yQTIzJOXSYpAlvIxcX67kXgiA8qWbNmoVCoWDJkiU1eu2qXr160bBhQ1555RWCgoJ0Cc6AAQMqlOAAfPTRRwBMnz79vglOZRFJjvBIUyokloQGYG9hzKWULGb/qB3K6ObmhkajYf/+/UyePJkXPF9g6jOTcejmwMJGlsQ9nwsGSpJ27iR55kyxcrkgCHrh5+eHWq0mJiamRq9d9cILL1BQUMAvv/zC+fPnadq0KR9//DFff13xdQS//fZbZFlm+PDhVRBpaSLJER55jpYmfBKqrc/ZcPQG359MxM/PjzVr1gDw8ccfs2nTJvo1GsCwhoMBeMfXkOmGKbwWH8/1zd+R+uFCkegIgiD8S/PnzycmJoa8vDxyc3M5deoU48eP1w3bf1SJJEeoEVp52/Nme+3z4KmRZ4lJyyYkJERXsT9kyBDOnTvHmy3GE+LehcIsNT/GZRJTWMDrCTdI+OorMj7/Qp9dEARBEKpZhZOckqmgBaG6je1Qn6e8bMktVDN6nbY+5/3336djx47k5uYSEhJCZmYm7z43n07eLXGf6IHSTMHp/HzeSkwkcfFibm/YoO9uCEK1EncwhZqqMn53K5zk1K1bl65du7Jx48Yyq9UKQlVSKiSWhgZib2H0V33OBQwMDNiwYQN169blypUrLFy4EAOFAQs7f8EzPj64j/dAYSTxR24O05KTSZo9h8ztO/TdFUGocoaGhgClVqMWhJqk5He35Hf536jwEHKFQqGbNdHS0pLevXszYMCAKp/QpyqIIeQ10+9X0hj49VFkGT4JDeCVgDocP36cdevWsWDBAl21fmZBJoMjXuTkiQTil1xHVsNrNjZMq10Ht2WfYdm2rX47IghVLDk5mTt37uDo6IiZmVm5Z+oVBH2SZZnc3FxSU1OxsbHBxcWlTJvyXr8rnOT8+OOPbN26lW3btnH79m3tQSQJDw8PBg4cyMCBA/H09Kxgl/RDJDk118e/XObTfVcxN1Ly45ut8XK497TgN7OTGRDxIhej0kj4PAEFEpvrutPI2pq6X63CrEWLao5cEKqPLMukpKQ8dOZbQXgU2djY4OzsfM/kvMqSnBJqtZp9+/YRERHBunXryMnJQZIkJEkiJCSEFStW6BY/e1SJJKfmKlZr6LfqCEdib+HnYkXkqGcwMdQOzywuLmbmzJkMGTKEevXqcS39PIO29yXm13SamBvxeVFHcn8/iMLCAvdv12By1wyigvA4UqvVFBUV6TsMQSg3Q0PDBw65r/IkR6VSsX79er755huOHTtWqkBIkiReeOEF3bTNjyqR5NRsN1X5dP3kdzJyCukXXJe53f0BGDduHEuWLKFJkyYcPnwYMzMzzsbtI2z/GPIkiZeKLRn+qxfZfx7DyM4O93XhGNeQu4+CIAhCFc54vG/fPvr374+LiwujR4/mzz//RKlU0qNHD/bu3UtycjL169fn119//U8dEISHcbIyYfFf61utOxLPj6eTAHj77bdxdHTkzJkzDBs2DFmW8fdoz+LAtzGQZX40yOLLFsn0SEnmtxvxxIeFUSRGDQqCIDx2KpzkdOzYkQ0bNpCXl0edOnWYPXs28fHxfPfdd7Rr1w4nJyeaN29OXl5eVcQrCKU85+PA6LbeAEzZepbY9Bzq1KnD5s2bUSqVrF+/nk8//RSAVk0H856ndkG+lTtOEa1SMTY5meMx14gfEkbxrVt664cgCIJQ+f7VZIAdO3Zk69atxMXF8e677+Ls7Fzq86VLlxIbG1spAQrCw7zVsT4tPWzJLijWzZ/Tpk0b3fooEyZM4PfffwegW5s5TLQJwDnUGYvGFuSp1byelMiFS5e4MXQY6uxsfXZFEARBqEQVrsm5evUq3t7eVRVPtRI1OY+PlMx8ui79nVs5hQx4yp33Xm2MLMv069ePDRs24OTkxIkTJ6hduzZoNCxe34lVuSnEfRhLbkweTkZGhLu6Uf+ZZ3D78gsUYtVmQRCER1aV1eQcPXqU8ePHc/HiRd22ixcvMn78eNavX//vohWE/8jZ2oRFvZsCsDbqOtvPJCFJEl9++SX+/v5kZ2dz4cIFbWOFgrd6/cCrSmPqjnPHpLYxNwsLGZaYQMLhwySOG48sRqIIgiDUeBW+k+Pn50dKSgqpqam6WQiLi4txcHDAxcXl7wtJDSDu5Dx+Ptx5ieX7Y7AwNmD7m63xsDcnJiaGgoICGv5jqHjR7Tje+q4re3M1xL1/jYKMIrpYW7PI2QWrl1+i9oIFSAqxvJsgCMKjpsru5MTFxVG3bt1S0ywbGBhQt25d4uLiKhzo+vXradasGaamptja2tKzZ0+uXLny0P1iY2MZPHgwLi4uGBkZ4eTkxIsvvkhmZmaFYxAeH+M7+RDkUUtbn7NeW59Tr169UglOyXwhhrU8+Kjzl7Qwk6k70QPbxla8v2IxGBig2vYjN+fOE+v+CIIg1GAVTnKMjY2JiYkhLS1Nty0tLY2YmBiMjY0rdKwvvviCfv36cfLkSVxcXFCr1URERNCqVSuSkpLuu190dDRBQUGsWbMGlUqFn58ftra27N69m6ysrIp2SXiMGCgVLO0bSC0zQ84nqZj308VSnx84cAAfHx9OnDgBgKl7K5a1mEZDW4nab9dltrQG6/emgyRxe9060j/9TB/dEARBECpBhZOcFi1akJeXR6tWrZg/fz7z58+ndevW5OXl0aICU+QXFBQwdepUAHr06MG1a9e4ePEilpaWpKWlMX/+/PvuO2bMGDIyMmjXrh2JiYmcPn2aixcvkpmZWWak1z/PqVKpSr2Ex4+LtSmL+gQA8O3h6/x09u85cBYvXkxcXBwhISFkZGQAYN1sICvrdse5uJjYggwmsp7dz7VidkoKacuWcWvNGn10QxAEQfiv5Ar65ZdfZIVCUeolSZKsVCrlPXv2lPs4f/zxhwzIgLx+/Xrd9k6dOsmAXL9+/Xvud+vWLVmSJBmQQ0JC5Hr16skWFhZycHCw/MsvvzzwnDNnztSd8+5XZmZmueMWao75P12U3SdtlxvP2CnHpWfLsqz9/alXr54MyJ06dZKLi4u1jdVqOWZdiNzqKz+5/of1ZUmh/R0bZmsnX2jgK9/eGqm/jgiCIAilZGZmluv6XeE7OZ06dWLjxo14eHggyzKyLOPp6cnGjRvp0KFDuY9z48YN3XtHR0fdeycnJwDi4+Pvud+VK1d0dRJbt25Fo9FgYmLCkSNHeOGFFzhy5Mh9zzllyhQyMzN1r7tjEB4/E573oYV7LbIKinlj/UkKitXUqlWLrVu3YmZmxu7du3n33Xe1jRUKvHqs4bNiK6ztDXEZqF319stbGay+dYvk6dPJ2rNHj70RBEEQKupfDR3p1asXMTExpKamkpqaSkxMDD179qzQMeT7FHSWbL/XqqOgHclVomPHjsTExHD16lVsbW1Rq9WsWLHivuc0NjbGysqq1Et4fBn+VZ9jY2bI2cRM5v90CYAmTZqwatUqAObPn09kZKR2B2MLAvps4WNVMQ5tauHUU5twf5iWyg+3bpE4bjw5hw/rpS+CIAhCxf3r8bGFhYUUFBSQm5tLfHy87lVedevW1b2/efOm7n1qaioAbm5u99yvTp06uvctWrRAkiSsra3x8fEB+FcjvITHV20bU938OasPxbHznLY+p2/fvowbNw6AQYMGcfnyZe0O1q48FxLOnFsq7F+0x66zHQDTb6aw7/Ytbox+g7wzZ6q/I4IgCEKFVTjJycrKok+fPlhaWlK3bl08PT11Ly8vr3IfJygoCDs77QUkIiICgMTERA7/9S/lLl26AODr64uvry+ffaYd5eLu7k79+vUBOH78OLIso1KpiI6OBtB9Jggl2vs6MaKN9ndz4pYzxGfkAvDBBx/Qpk0bXnzxRVxdXf/ewS2Ilzt+xNu37uDcxxmbVjaoZZnxKSnE3rnDjWHDKbh6VR9dEQRBECqgwknO9OnT+e677ygqKtLV5Nz9Ki8jIyPmzZsHaGtrvLy8aNiwIdnZ2djb2zN58mQALl++zOXLl0lPT9ftu2DBAiRJYvfu3Xh7e+Pt7c2tW7cwNzdn/PjxFe2S8AR4+/kGNHevRVZ+MW9sOEFBsRpDQ0O2b9/O+vXrMTc3L71Dk94MbjKMwVlZ1PlfHawCLAkdGUqDFs1RZ2YSPySMwoQE/XRGEARBKJcKJzk//PADkiQxbdo0AOrVq8fIkSOxtbXV3W0pr+HDhxMeHk5AQABJSdpp+ENCQjh06JB2jaH7CAkJ4fvvvycoKIikpCQUCgWvvvoqx44dw8/Pr6JdEp4Ad9fnnEnIZMHP2vocCwsLXf2XLMulC9fbv8t4h1a8nJeL25i6nH/qElnzx2Jc35vi1FTtyuV3zRclCIIgPFoqvKyDsbExdevW5cqVKygUCoKDgzl8+DD16tWjffv2fPnll1UVa6UTyzo8efZevEnYmmMArOzfnC6NtfMqFRYW8tprrxEZGcmuXbvo2LGjdoeCbIq+7swYbvKHmSnWRlasbLSAuV378rqJKa6NGuH+7RqU1tb66pIgCMITp8qWdTA2NsbS0hIAExMTEhISKCoqoqCggO++++7fRywI1aCDnxPDn9PW57yz5TQ3bmnrcwwNDbGyskKj0RAaGsr169e1OxhbYNh3Ix9nQ5P8AjILVXR6ozcbExMZnpxE2oUL3Bj5OprcXH11SRAEQbiPCic5zs7OJCYmAtpHVUlJSTg4OJCUlFThZR0EQR8mdm5AYF0bVPnFvLH+BIXFGiRJYtmyZTRv3pyMjAx69OhBfn6+dgcbN8xC17Ms/Q6ehUVYvWSFcS1jLufmMjolmVvHj5MwZixyYaF+OyYIgiCUUuEkJzg4mNzcXM6cOcPgwYN1o5sABgwYUOkBCkJlM1Qq+LRvINamhpxOyOSDndr6HFNTUyIiIrCzs+P48eOMGjXq72J6t5bYdFvK5ympuNoqcBvvhqG5Icezs5lwM4U7v/9O4juTkNVqPfZMEARBuFuFa3L+af369URFRdGkSRPCwsLuO4nfo0jU5DzZ9ly4ydBvtfU5XwxozvONtPU5u3fvpkuXLmg0GlauXMmIESP+3mn3TK4eXcbA2k6kXMkj/uN41IVqXraxYZ6jE7a9e+M8Z3aN+nMgCIJQ05T3+l2hJKeoqIgRI0ZgYmLCsmXLavxf5CLJEd7ffoFVf8RiZWLAjjHP4mZrBmjn0Jk8eTLm5uZcv35dN6cTGg1s6sfJuD0Mc3Ei7XQWNz69gayWGWpnx3h7B+yGDcVxwgQ99koQBOHxViWFx4aGhmzevJnDhw/X+ARHEADe6eJLgJu2PufNDScpLNZot7/zDqNGjWL37t1/JzgACgWEfEGgjQ8f3UynVlMrag+pjUUtC/q+/TYAGV+uIr0GjTIUBEF4XFW4Juf5558nNjaWzMzMqohHEKqVkYG2PsfKxIBTN+7w4V/1OSWFyE8//XTZnYwtoe8G2iosmJmeQa1WtXCb60bCS7VxnDgRgLSPF3F70+bq7IogCILwDxVOcp5++mny8/N56qmn+OCDD1izZg3ffvut7iUINY2brRkf9dKub7Xqj1h2X7hZps2ZM2f48MMP/95gUxf6rKN7XhFjb91BaaZk4bGFHG7nyPUundmTlUXKrFmofvqpurohCIIg/EOFC48VCsV9H1VJklRqlfBHnajJEe4258cLfH0wFmtTQ3aMaY1rLW19TnJyMvXr1ycnJ4cNGzYQGhr6906nNiB/P5IPbW0It7aiKLGI+LnxqAsKWelSm6dsbHBbvgyLZ5/VU68EQRAeP1U2GSBwzzWrZFlGo9H864AFQd8mv+BLU1drMvOKeHPDSYrU2t9nFxcXxowZA0BYWBhnz579e6eAvkitxjLx1h265uRh4GKASSMTCtVq3ryZwrksFQlvjiH3xAl9dEkQBOGJVuEkR6PRPPAlCDWVkYGCz15rhpWJASfj77Bw12XdZ++99x6dOnUiNzeX7t27c+fOnb937DAThc8LvJ+aRqtiGZfhLlg3tianqIiRN29yTZXJjREjyb90qfo7JQiC8AT7V3dyBOFx5WZrxsK/6nO+OHCNvRe19TlKpZINGzbg7u5OTEwM/fv3/zupVyihx5cYOjZicVIiTZSG1B5dG6t6VtzKz2fYzZsk3bpF/NBhFMbF6alngiAIT54K1+S0b9/+/geTJPbu3fufg6ouoiZHuJ/ZP57nm4Nx2JgZsmPMs9SxMQXgxIkTtGrVivz8fGbNmsXMmTP/3un2dfiyPbfybzHIox5XM3NI+CCB7MRsvC0t+dbJGQc3N9w3rMfQyUlPPRMEQaj5qmQyQPi78Pju3Up+liQJdQ2a1l4kOcL9FBZr6LnyEGcSMmlW14ZNI57GUKm98blmzRoGDx7M888/z08//YRSqfx7x+uHYc1LJCo0DHCvR2KaioR5CTwX+DQLZQllQgJG9erhHr4Wg1q19NQ7QRCEmq3KkpzBgweXGl2VmZnJ/v37ycrKIjQ0lLVr1/77qKuZSHKEB4nPyOXFT38nK7+YEW28mPKCn+6zH374gW7dupVOcEqcXAc/jOKyoSH/c/cg42YWHfw78LHfZBL6D6T45k1M/P2p+803KC3Mq7FHgiAIj4cqS3LuJT09nSZNmjBo0CDmz5//Xw9XbUSSIzzMznPJjAzXjoz6enAL2vve+zFTcXExBgYGf2/4ZToc+pTjZpYMd3GgUFNMSP0QJjv05/OuXemoUGL59NO4fb4ShbFxdXRFEAThsVGlQ8j/yd7eHm9vb1avXl0ZhxOER0aXxi4MfsYDgAmbT5N0J6/U5wUFBYwcOZJBgwaVeoRLx9ng04XmuVksvJ2HAgVbr2yl87thjLt6lfm3Msg5fJjECROQa9DcUoIgCDWJwcOblDZnzpxSP6vVaqKjo/njjz+oJWoMhMfQlK6+HL9+m7OJmby54SQbhz+lq885c+YMX331FcXFxbRs2ZKxY8dqd1IoIeRL+Loz7VMvMMPMl1mGucTZxiFJEuvT06mlUDB6z16Sp7+Ly7y5SAox2FEQBKEyVdqMx7IsM2DAANasWVNpwVU18bhKKK/4jFxeXPo7WQXFjGxTj8kv+Oo+W7p0KWPHjkWpVLJ3717atGnz9463r8OX7SA3gy/qB/NpcTK39t0i6dskAKY6O9Pf2oZaAwfgNGWKWPhWEAShHKqsJsfDw6PUX8SSJOHo6EiHDh2YMmUKFhYW/z7qaiaSHKEifjqbzKh12vqcb/4XRLsGjsDfCf66detwdHTkxIkT1KlT5+8d/xpxJWuKmO/fgQ3ZV0jflk7K1hQAFrrU5kUrK+zHvInDqFHV3i9BEISaploLj2sqkeQIFTXjh3N8e/g6tcwM+Wnss7hYa+fPyc3N5emnn+bMmTM89dRT7N+/H+O7C4pPhsMPo9EA7zR7gZ23zpG2Po3U3akYKJUsc3HhWXMLnKZNw3ZAf/10ThAEoYaossLjgoICVCpVqYU4i4uLUalUFBQU/LtoBaGGmNrVj8Z1rLidW8SYDScp/mt9KzMzM7Zu3YqNjQ1RUVGMGzeu9I6B/eHpN1AA8878ylN2jXDo64DDMw7IQPFfk2zenDuXzG3bqrdTgiAIj6kKJzmvvvoqtra2REdH67ZdvXoVOzs7unfvXqnBCcKjxsRQyWd9m2FhbMCfcbdZtPvvPwf16tVj/fr12Nra8uKLL5bdudMcqN8Zo+J8Pok+RSNbHxyGONB8dnN6Ll9IrYEDAEiaMpWsfb9WV5cEQRAeWxV+XGVvb18myQFo0KABt27dIi0trVIDrEricZXwb20/k8Qb608CsGZIS9r4OOg+U6lU9/99ylfBV89D2kUyXJow0N6C+OwEfGr58PXzXxE7YTrpO3/G1dwCty+/xDy4ZXV0RxAEoUapssdVWVlZpR5VlSgqKiIrK6uihxOEGqlbk9oMeModgHGbTpGSma/77O4/cHFxcaSnp/+9o4kV9N0AprbYJZ/h82Ib7E3tib4dzZANYYT8/BPDMjJIz80lYdQo8s6dr7Y+CYIgPG4qnOS4ublx/fp1Fi9erJv8bMmSJcTFxeHq6lrpAQrCo2rai340dLHiVk5hqfqcEvv376d58+b07du39Jputp7QJxwUhrhe/IkVNkFYGFpwPvM86bnpxGVmMvL2LVQqFTeGDaPg2rVq7pkgCMLj4V/V5MiyzNtvv425uTlmZmZMmDABSZIICQmpihgF4ZFkYqhkWT9tfc7RuFss2XOl1Of29vbk5+ezZ88epk+fXnpnj1bQbREAvoc+Z6lnL8ztzHEe54yZjRnnb91iTOYdcjMyiB8SRlFiYnV1SxAE4bFR4SRn1qxZBAYGIssy+fn55OfnI8sygYGBzJgxoypiFIRHlqe9OfND/AFYtv8qB6L/rklr3LgxX3/9NQALFiwgIiKi9M7NBsJTowEI2vshHzQegYmzCS5vuWBibkJUWhrvqDLJT04mfkgYxRkZ1dMpQRCEx8S/mienqKiIjRs3cuTIEQCCg4MJDQ3F0NCw0gOsSqLwWKgs0yLPsu5IPHbmRvw09lmcrEx0n02YMIFFixZhYWHB0aNH8fP7ezVzNGpY3weu7gZLFzZ3nMB7p5aSfTGbhEUJFBcV08vFhVmWVpg0bIj7t2tQWlrqoYeCIAiPDjEZYDmIJEeoLPlFarovP8TFZBXBnrasGxqMwV/rWxUXF9OpUyf279+Pr68vR44cKf37lp/514irS1A7kBUtQlh+9ktUx1UkLEvA28ODtXb2WKpUmDZvTt1VX6IwNdVTTwVBEPSvykZXTZw4kWbNmnHq1CndtjNnztCsWTPeeeedfxWsINR0JoZKlvdrhrmRkiOxt1i69+/6HAMDAzZt2kSdOnW4dOkS8+fP/8fO1tB3I5jaQtJJRl49QZ8GfbBqboX7G+588sMymny7BoWlJXnHj5MwdixyYWE191AQBKHmqfCdHDc3NzQaDYn/KIR0dXVFkiRu3LhRqQFWJXEnR6hsP5xKZOzGU0gSfDukJc/W/3v+nCNHjvDNN9+wZMkSTExMyu4c9wd8+wpoilG3mcxEKY3d13djbmjO152/xuN6Pkf7D8BJo8Gqa1dqL/wQSamsxt4JgiA8GqrsTk5qaiq1atUqs93GxqZGTQQoCFXhlYA69G1ZF1mGtzaeIlX19/w5wcHBrFy58t4JDoBHa3hRO+JK+dsCFtg9TUvnluQU5fD6ntf59I/ddL56hX15eah++omU99/nCX7aLAiC8FAVTnJKZjsuKToGOHr0KJcvX75n8iMIT5qZLzXE19mSjJxCxmw8iVpTNhFRq9XMnTuXuLi40h80HwRPaVciN/rhTT7xC8PX1peMvAw+/f5TCouKmJCcxLG8PO5s2EjaJ59UQ48EQRBqpgonOe3ataO4uJg2bdrQuXNnOnfuzHPPPYdGo6FDhw5VEaMg1Cgl8+eYGSmJunaLT/ZeKdNm8uTJTJ8+nR49epCXl1f6w07vgXdHKM7D4rswVjw1GzdLN2z62+DS0oWCoiJGp6VyMT+fjJWfk/H1N9XUM0EQhJqlwjU5V69eJSgoiMzMTCRJAkCWZWrVqsXRo0epV69elQRaFURNjlCVvj+ZyFubtPU54WHBtPK2130WHx9P8+bNSU9PZ/DgwXz99de6P0+AdsTVqk6QfhlqN+NGzy/ov3so6ap0bn16i5SzKThYWfGtnT3uRka4vP8eNj176qGXgiAI1a/KanK8vb05duwYgwcPxs/PDz8/P/73v//VuARHEKraq4F16NvSDVmGsRtPkZr1d31O3bp12bRpEwqFgtWrV/P555+X3tnEGl7bCKa1IOkEbvvms7LDCizNLan1ei0c6jmQplIxIvMOacXFJM+YiWrXL9XcQ0EQhEebmCdH3MkRqlB+kZpXlx3kUkoWT3vZET40GKXi7zs2Cxcu5J133sHQ0JDffvuNp59+uvQBYn+Hta+CphjaTeeIz3O8vud18m7nkfphKrcSbzG9QwdeS0hEMjTEdeUKLFq1qt5OCoIgVLMqnQywsLCQgwcPkpSUVHrhQWDgwIEVj1ZPRJIjVIerqdm8/Nkf5Baqeatjfd7q6KP7TJZlevfuzZYtW6hduzbHjx/H2dm59AGOfQPb39K+7/0tu8xMmPjbRArSCvBP8Wfzgo0kvT2RrJ07kczMcP/6K0wDAqqtf4IgCNWtypKcK1eu0KlTp3vOhyNJEsXFxRWPVk9EkiNUl8iTCYzbdBpJgnVhwTxzV31OVlYWTz31FNeuXWPr1q288MILZQ/w8yQ4shIMzeB/P7NRdYm5R+YC8O5T79LT81WuDR9B7uHDGNvY4P7tt5g08Cl7HEEQhMdAldXkTJ48mfj4eGRZvudLEISyuge60qeFtj5nzD/qcywtLYmMjOTQoUP3TnAAnp8L9TpAUS5s6EtonbaMaDICgPej3mf71Z2MSrjBzIJ8iu/cIX5oGIU1aGJOQRCEqlDhJOePP/7AwMCA3bt3AxAYGMiGDRuwt7fXbRMEoaxZLzeigZMl6dkFjNt0qtT8OT4+PgQGBup+/udjYJQG0PNrsPeBrCTY+BqjGw2hp09PZGQmrJvAvv37+f76dT4uKqI4NY34/w2h6GZqdXVPEAThkVPhJOf27dv4+fnRoUMHJEnC0NCQPn364OzszLx586oiRkF4LJgaKVnWLxBTQyUHr2aw7Ner92x39OhRGjVqxJkzZ/5xAJu/1riqBYnHkba9yfSW0+hQtwMmPiZ4DPcA4JvYa3ylUVOUkMCNoUNR37lTpf0SBEF4VFU4ybG0tESj0QBgYWHBpUuXOHLkCPHx8Rw+fLjSAxSEx4m3oyXvv9oYgCV7ojkUk16mzdy5c7l8+TLdu3fn9u3bpT+0qwe9vwWFAZzbgvLgYj547gOaOzXHNNgU70HeACy6epUtxcUUXLlC/IgRaHJyqrxvgiAIj5oKJzl169bl+vXrqNVq/P39ycrK4plnniErKwsXF5eqiFEQHis9mrvSq7krmr/mz0nLKij1+TfffIOnpyfXrl2jX79+un9U6Hg+B10Xat/vex/jy7tY2n4pPrV8MGlnQr0e2vmqZl6LYbe6mPzTZ0h48000YuVyQRCeMBVOcvr160ebNm2Ijo5m2rRpGBoaIssyCoWCWbNmVUGIgvD4mfNKY3ycLEjLKlufY2trS0REBCYmJvz888/Mnj277AFaDIGW2sJjIkdglRHLyo4rqWNRB5NuJnh18UKWZd7LyCDX2JicQ4dJmvA2cg0a/SgIgvBf/efJAGNjYzl58iSNGjWiQYMGlRVXtRBDyAV9unIzi5c/O0hekZoJnXx4s0P9Up+vXbtWN+/Utm3beOmll0ofQF0M63tBzD6wqgPDfuW6nM/AnweSkZuBOkLNmllraFSk5sbwEchFRVj3CMHl/fdLLyEhCIJQw1TZEPJ/8vT0JCQkpFSC8/TTT2NgYPBfDy0Ij7X6Tpa891d9zuI90URdyyj1+YABA3jjjTd0769c+cdCn0oD6PkN2NUHVSJsfA13UyeWd1iOuZE5Br0M+C7nO0yfCqb2oo/RSBKZEVtJ/XChmO5BEIQnwn9Ocu5H/CUqCA/Xs7krPZpp63PGbDhJenbp+pyPP/6YVq1a0aZNGxwdHcsewNQGXtsEJjaQeAy2vUkju4YsbrcYA4UBO+N2suDoAs6amRGan0dyURG3vvmGjM+/qJb+CYIg6FOVJTmCIJTPe682wtvRgtS/6nM0d9XnGBkZsWPHDiIjI7G2tr73AUpGXElKOLsZ/ljEM7WfYV7reUhIrL+wnteGvca5uDhez8/jjlpN2pIl3N6woZp6KAiCoB8iyREEPTMzMmB5v2aYGCr4/Uo6K36LKfW5tbU1CoX2j6osy5w6darsQbzaQNcPte/3zoGL23nB8wUmtZyEpJAwGWqCnbMd0UlJvFFUSI5GQ8qc98jcvqOKeycIgqA/IskRhEeAj5Mlc17R1ud8/MtljvyjPgegqKiIgQMH0qJFC/bv31/2IEFDIWiY9v3W4ZB8hn5+/RjmPwwjOyPsxtphVcuKE9evM0GjplCtJmnyZLJ/+60KeyYIgqA/IskRhEdEr+auhDSro63P2XiSjH/U5xgYGCBJEmq1mt69e5OQkFD2IF0WgFdbKMqBDX0h6yZvBr5JSP0QjFyMcBnjgqmZKQdiYnhXklAXFZEwZiy5x45VTycFQRCqUZUkOWKxTkGoOEmSeO+VxtRzMOemqoBxm0+Xqs+RJImVK1fStGlT0tLS6NGjBwUFpRMhlAbQazXYeYMqATb1Qyou4N2n3qWtW1sMPQ3xGOOBoaEhP0ZfZou9HXJBATdGvk7+hQvV22FBEIQqVuEkZ8iQIcydO7fM9q1bt7J8+XIAoqKiys7SKgjCQ5kbG7C8X3NMDBUciE5j5YHS9TlmZmZs3bqVWrVqcfToUcaOHVv2IKa1oO8mMLGGhD/hxzEYSEoWPreQZo7NUPoq8RntQ9dXuvLOjh2YtWiBJjub+KHDKIiNraaeCoIgVL0KJzmrV69mx46yxYoLFy7kzTffrJSgBOFJ1sDZkjkvl9TnRPNn3K1Sn3t5ebF+/XokSeLzzz/nq6++KnsQe2/otUY74urMJvhjMSYGJixtvxRvG2+kAAlpoES+iQbXFcsxadgQ9a1bxIeFUZScXB3dFARBqHLlTnLi4+OJj48HoKCggBs3bui2Xbx4kevXr4tZVAWhkvRq4Ur3wDqoNTJvrj/JrZzS60516dKFOXPmADB27FjS08su9Em9dvDCB9r3e+fApR1YG1uzsuNKapvX5nrWdUbvGU2+sYLlrnX42diI4qRk4oeEUXzrVtnjCYIg1DDlTnI8PT3x9PREkiROnTqFh4eHblvjxo25efMmrq6uFQ5g/fr1NGvWDFNTU2xtbenZs2fZmV3vQ61W8/TTTyNJEpIkMXny5AqfXxAeRZIk8f6rjfFyMCdFlc/4zaXnzwGYOnUqYWFh/Pjjj9jb29/7QC2HaUddIUPEMEg5i5O5Eys7rcTG2IZzGed4edbLfPTpp0y6cIGDRkYUxsZyY+gw1NnZVd9RQRCEKlTuJOfuYuKS93e/DAwMmDp1aoVO/sUXX9CvXz9OnjyJi4sLarWaiIgIWrVqRVJS0kP3nzNnDlFRURU6pyDUFNr6nGYYGyjYfzmNzw9cK/W5QqFg1apVtGvX7sEH6rIAPNv8PeIqOxVPa0+Wd1iOqYEpN+vfxK+TH8XFxYy5Es1pQwPyL1wg4fVRaPLzq7CHgiAIVavcSc6vv/7Kvn37kGWZhg0b8uuvv+peR44cITk5meHDh5f7xAUFBbqkqEePHly7do2LFy9iaWlJWloa8+fPf+D+hw4dYu7cufTq1atC51SpVKVegvAo83W2YvbLjQD46JfLHIu7/2OkS5cu8cknn5T9QGkIvdeAbT3IvAGb+kNxAf4O/ixuuxhDpSGKPgoaPN2AvPx8Xo+P54pSQe6ff5I4bjxyUVFVdU8QBKFKlTvJadOmDW3btmXmzJmMGzeONm3a6F5BQUHY2tpW6MTHjh0jI0M74VmPHj0AqF27Nk899RQAu3btuu++KpWK/v37U7t2bb74ovxr8MyfPx9ra2vdy83NrUIxC4I+9Aly45WA2tr6nA0nuf2P+hyAlJQUgoODeeutt9iyZUvZg5jW+muNK2u4cQR+HAuyTKs6rXiv9XtIBhLKwUrqBdTjjkrFiNRUEiXI/vVXkqZNQxajJQVBqIEqPLpqwIAB1K9fn/T0dGRZ5uOPP+aVV15hxowZFFXgX3w3btzQvb974UEnJycAXZHzvYwePZrr168THh6OjY1Nuc85ZcoUMjMzda+7YxCER5UkSczt7o+XvTnJmflM+O50mfocZ2dnRowYAcD//vc/Ltxrzhv7+to5dCQlnN4AB7V3fbp5dWNii4kojBUYDTOirk9dUtLTeT0nh0KFAtW2H7k5d56Y+0oQhBqnwknOuHHjaNeuHWlpaaxfv56JEyfy448/MnfuXN59991yH+d+f2GWbL/fSK3IyEjCw8OZOnUqzz33XIViNzY2xsrKqtRLEGoCC2MDlv1Vn7PvUipf/n6tTJt58+bRvn17srOz6d69O5mZmWUPVK+9tkYHYM8suPQTAAMbDWRI4yEozZVYjLLAzcuNOR98gMcHH4AkcXvdOtI//awKeygIglD5KpzknDhxAgcHB/z8/NixYwdKpZIhQ4YgSRIRERHlPk7dunV172/evKl7n5qaCnDfR0mnT58GYNGiRVhYWGBhYaH7bNGiRf9qhJcg1AR+LlbMfElbn/Phrsscv166PsfAwICNGzfi5uZGdHQ0gwYNuveknC2HQYshgAxbh0HKOQDeavYWr9R7BYWNAvt37fHt4Iv1S91wenc6AOnLl3NrzZoq7aMgCEJlqnCSk5aWRp06dQA4d+4cQUFBrFq1ioYNG5ZrRFSJoKAg7OzsAHTJUWJiIocPHwa084AA+Pr64uvry2eflf5XZG5uLjk5OeTk5Oi2FRUVkS2GvQqPsb4t3Xi5aW3d/Dn/rM9xcHAgIiICIyMjfvjhBxYsWFD2IJIEL3wIns9BYTZsCIXsNCRJYtYzs2jj2oYiqYjRe0dz9fZV8tq04VPXOhTLMjfnL+BO5PfV01lBEIT/qMJJjoWFBcnJycTHx3PlyhUaNmwIgEajwdjYuNzHMTIyYt68eYB2SQgvLy8aNmxIdnY29vb2ujlvLl++zOXLl3WTnc2aNavM8PUSkyZN4s6dOxXtkiDUGJIkMS/EH097c5Iy83n7u9NlHv0GBQXpllj55ZdfKC4uLnsgpaF2RmRbr1IjrgwUBixss5CmDk3JKsxi2K5htG3flhV79zLP0gJZlkmePp2sPXuqo7uCIAj/SYWTnKZNm3Lz5k08PT0pLCykVatWaDQabty4gbu7e4WONXz4cMLDwwkICCApKQlJkggJCeHQoUPUrl27oqEJwhPBwtiAz14LxMhAwd5Lqaz6vex6U2FhYWzcuJHdu3djYGBw7wOZ2WrXuDK2hhtR8ONbIMuYGpiyrMMy6lnXI70gHZvuNigUCjYeP85ntWqBWk3iuPHk/HXXVRAE4VElyRUcMnHkyBFefPFFbt26xdNPP82+ffs4ePAgHTt25PXXX2fZsmVVFWulU6lUWFtbk5mZKYqQhRonPOo6078/h4FCYvPIp2lWt9YD22s0GhSKe/y75upeWNcLZDV0mgOttIt+puSkMODnAaTkpGB+0pwjnxwBYNozreiXkYFkZob76m8wbdKk0vsmCILwIOW9flc4yQHtCKjbt2+XmhtHrVajVCr/XbR6IpIcoSaTZe28OdvPJFPHxpQdY1pjY2ZUpl1RURGTJk0iIyOD1atX33vk4pHP4ed3AAn6boAGLwBw7c41Bu4cSGZBJhYHLYj6UjvD+IetWtEtPQOltTXu68Ix9vauyq4KgiCUUt7rd4UfV4G2LuDOnTts2LCBH374AaDGJTiCUNNJksT8EH887MxIvJPH29+duefUDKdOnWLp0qV8++23ulqdMloOh+b/Q7vG1VC4eR4ALxsvlnVYhonShOxW2QT3DQZg8uHDHHKwR52ZSfyQMAoTEquqm4IgCP9ahZMctVrN0KFD8fHxoX///nzwwQesXbsWpVLJp59+WhUxCoJwH5Ymhnz2WjOMlAr2XLzJV3+Urc8JCgrigw+0q5G/9dZbHDx4sOyBJAm6LgSPZ/8ecZWjLfZv6tCUj9t+jFJSkv18NkEvBeHr60vHlSsxru9NcWoq8UOGUJyWVqV9FQRBqKgKJznz58/n66+/RqPR6P7V2L17dwwMDNi2bVulBygIwoM1rmPNuy9pRzku+PkSJ+Nvl2kzfvx4evfuTXFxMT179iQ5ObnsgZSG0PtbqOUJd+J1I64AnnN9jjmt5iBJErmv5jL6y9G4N2qE26qvMKxTh6L4eOKHDkN9rwkIBUEQ9KTCSc4333yDoaEh33//vW6bhYUFbm5uXLx4sTJjEwShnPoH1+VFfxeKNTJvrD9JZm7pJVYkSeKrr76iUaNGpKSk0KtXLwoLy66BhZmtdo0rYyuIPwzbx8Nf/5h5ud7LTGg+AUkpseLKCrbFbMPQyZFjPXtww8KcgsuXuTHydTS5udXRZUEQhIeqcJKTkJBAw4YNefnll0ttL1k9XBCE6idJEvN7+ONeUp+zpez8ORYWFkRGRmJlZcXBgweZMGHCvQ/m0AB6fgOSAk6Fw+G/J+Ic3HgwgxoOAmDGwRnMWj6L10aPZnhKCmkmJuSdPEnCmLHI90qgBEEQqlmFkxx7e3tiY2N1K4iDdjHNixcv4uDgUKnBCYJQflYmhiz7qz5n94WbfH0wrkyb+vXrEx4ejqWlJW3atLn/wep3hM7ayTr55V2I3qX7aHyL8bzk9RJqWc338ve4erhyPTGR13OyURkakvPHHyROmoSsVldyDwVBECqmwklO586dUalU+Pv7A3DhwgWaNWtGUVGRbikGQRD0o3Eda6Z38wNgwc8XOXXjTpk2L730ErGxsfTs2fPBBwseCc0GATJsCYOb2pXNFZKC2a1m07pOa9QWahzHOeLo7MiFq1cZC+QplWT9vJOUWbPFyuWCIOhVhZOcuXPn4urqSkpKCqAdq37r1i1q167NnDlzKj1AQRAqZsBT7nT1d6ZILfPG+hNk5hWVaVOybhxo14y7detWmTbaEVcfgfv/27vvsKiu9IHj36ENfegiiIKKEiyxiy12sUIsicYYjenJmrLJ+ktM0WRTzO6apimmrDEboykaYywJ9g72XlGa9CK9z8z9/TEyQAAdEaT4fp6HJwP33Ms9TC68nvOe8w6EklxYNdW44srSzJIPBn9AV7euFDsW4/cPPzROGg6cPsUrtraUqlRk/fILaR9+WG/9FEKIG7npICcyMpI1a9bw5ptvMnbsWMaMGcOCBQuIiIjAycmpHm5RCHEzVCoV70/uSmsXW+IzC/m/avJzyuzbt48ePXowffp0dNVNL1lYwdTvK6y4egi0hnwbW0tbPhv+GX4aP/Ld8un6cldsbGzYcvgQb7u5olcUMr7+hvSvv67P7gohRI1uOsgZMmQIL7zwAvPnz2fDhg1s3LiRBQsWMGXKFNk1WIhGomJ+TtiZFJbvj6m2nb29Pbm5uYSFhfHmm29Wf7FKK672w8a/G1dcOVk78eWIL/Gw9SCzZSZ9X+mLhYUFvkFBtJg7F4C0Dz4k8+ef66GXQghxfbXa8bi6fxXm5+fL/LsQjUiXVhpeHRsAwHubznGimvycu+++m6+vjbS88847xh3Mq6i44urYCggvr1HX0r4lX474EgcrB9LbpDPlyym896/3cHvsUVyfeAKA5AVvkvPHH3XbQSGEuAGTg5xhw4YxbNgwwJBsXPb5sGHD6Nu3L6dPn0aj0dTbjQohbt6s/r6M7nQtP2dV9fk5Dz74IM899xwAM2fO5OLFi9VfzH8EjHrX8HrLG3Bxs/FQe+f2fDb8M9Tmak6bn+at/YakY4enn+J4z56gKCT838vk7dlT530UQoiamFyg08zMzFjYr6ZTpkyZws9NaFhaCnSKO0F2YSnjl+zhytVCRnfy5IsZPaoU6SwtLWX48OHs2bOHwMBADhw4gL29fdWLKQqsfw6O/g+sHOCxLeBxl/Hwzis7eWHHC+gUHQ/5P0TYG2Hs2LGDL8aO5Z5Ll1FZW9N62X+x7dGjnnsthGjO6rwK+ezZswH47rvvcHd3Z+zYscZjtra2BAQE8Mgjj2BnZ3eLt377SJAj7hQn47OY/MV+SnUKb4V0YlZ/3yptkpOT6dGjB0lJSfzf//2fsd5VFdoS+H4ixO4Fpzbw+A6wK1+ttTZyLfP3z0dRFDz+8GDHzzuwtLTk2xEj6BEVjZmDA22+/x/WAQH11FshRHNX50FOGV9fX3r27MmaNWtu+SYbmgQ54k7y7b5o3lp/FitzM9Y83Z8urapOL+/fv58vv/ySzz///Pr/YMnPgG+GQWYMtBkAD/1mWIl1zX9P/ZePj36MoldwXu3M3k17sbOzY8WAAXSMjcPczQ3fH1Zg1aZN3XdUCNHs1VuQ05xIkCPuJIqi8NSKI4SdSaG1iy0bnhuIo7Vl7S+Yeg6+GWnYQ6fHTJiw2LC3zrXv9e9D/2bFuRWYac2w+s6Kw3sO4+riwspu3fFJSMDSy4s2q1Zi2aJFHfVQCHGnMPXvd61WVwkhmh6VSsW/J99NK2cb4q4W8Mqak9ddEakoCh9++CFXrlypvoHHXTBlmWHF1dH/QcQXlb7X3N5zGes3Fr2FHv0sPZ26dyLj6lUePX+ONA93ShMTiXvkUbSZVaumCyFEXZAgR4g7iMbWkk+n98DSXMWmU8l8HxFbY9vXXnuNl156icmTJ1NUVFR9ow6jYOTbhtebX4PILcZDZioz3hnwDv29+lNiUYL9U/a069COvIICzOfOxaJFC0ouX+bKE0+iy8uvy24KIQQgQY4Qd5xuPk68MsawIuqdDec4nZBdbbsnnngCFxcXDh06ZFxiXq1+f4PuM0DRw+pHIPW88ZCluSUfDfmIzq6dyVfn4/V3L37981cGTJhA62X/xdzJiaJTp4ifMwd9cXGd9lMIISTIEeIO9MgAX0YGtqBEp+dvK4+SW1R1/xxfX19WrVqFSqXi66+/Nm4aWIVKBeM+gtb9oTjnWo2rDONhW0tbPhvxGb6OvmTaZLI4eTHZxdmo27Wj4NV5aK2tKYiIIOGll1C02vrqshDiDiRBjhB3IJVKxaIpd+PtZENsRgGv/Hqq2vycUaNG8e67hg0A58yZw8GDB6u/YFmNK6c2hhVXP8801rgCcLF2YenIpbjbuHMp6xLPbX+OrTu2Mvihh5jv4oLe0pK8rdtIev0NFL2+ProshLgDSZAjxB3KkJ/THQszFRtPJrHiQFy17V555RUmTpxISUkJkydPJjU1tfoL2rkZalxZORj20Nn0D2ONKwBve2++GPEFDpYOHE09yqeHP6WwsJDfdmznQ982KGZmZP/2Gynvvy8lYoQQdUKCHCHuYN1bO/PKGMOmfG9vOFttfo5KpWL58uV07NiRlJQUIiIiar6gx10w5b+ACo5+BweWVjrc0aUjS4YvwcrMisselwmZH4JKpWLZhg1827kTAJn/+570L76o5uJCCHFzJMgR4g736EA/RtzlQYlWz5wa8nMcHR1Zu3Ytu3fvJiQk5PoX7BAMo66tuAp7FSK3Vjrcs0VP/jP4P5ipzLjgc4H75t0HwKJffuG3Xj0BSF+8hKsrfrj1zgkh7mgS5Ahxh1OpVCy6z5CfE5NRwLwa8nPuuusugoKCjJ/rr5c7028OdCtbcTUb0i5UOjys9TDmB80H4GyHs9z3nCHQefWHH9jRtw8AKe+8Q/bvv99q94QQdzAJcoQQONlasfgBQ37OhpNJrDxYfX5OmRMnTtCtWzdOnz5dfQOVCsZ/CK37GVZcrZwKBVcrNZncYTLPdn8WgHPdzxH6cCgAG9LScJoxA4DEea+Su33HLfZOCHGnkiBHCAFAzzbOvDzakJ/z1vqznEmsfv8cgDfeeINTp04xadIksrKyqm9koYapK8CpNWRGG1Zc6SpPhT3e5XGmB0wHFcQMieGld19i7dq1eL46D01oKOh0JLzwAvkHaljVJYQQ1yFBjhDC6LFBfgwPKMvPOUZecfX71vz3v/+ldevWREZGMnPmzJqnruzc4IGfwMoeYvbAprmVVlypVCpe7vMywb7B6NCxu81uLuVeQmVmhuc7b1MUFIRSUkL8M89QePpMfXRZCNGMSZAjhDAqy8/x0lgTnZ7PqzXk57i7u7NmzRrUajXr16/nvffeq/miLQJh8rUVV0e+hYNfVTpspjLjvYHv0bdlXwq1hTy99WmiMqN4ce5cxm0OIz0wEH1+Plcef5ziqKg67rEQojmTIEcIUYmznRVLpnfH3EzF7ycS+fFQ9QU6e/XqxRfXlnrPnz+fP/74o+aLdhwNI98yvP7zFbi0rdJhK3MrPhn6CYGugWQWZ/LExifYvHUzSUlJzD5xnNz27dFlZhL3yKOUJiTUST+FEM2fBDlCiCp6tnHh/4I7ArDg9zOcTcyptt3s2bN56qmnUBSF6dOnc/ny5Zov2v85uHu6YcXVL7Mh7WKlw3aWdnw+/HNaO7QmRZeC70u+tG7TmstRUTweG0Nx69Zok5MNlcszMmr4JkIIUU6CHCFEtR4f1JahHd2N++fUlJ/z8ccfExQURO/evXFycqr5gioVTPgYfIKgONtQ4+ovK65cbVz5cuSXuNm4ccXsCj3n98TDw4OTp0/zXG4OuhYtKImNJe6xx9Hl5tZdZ4UQzZIEOUKIapmZqfjg/m601FgTlZ7Pa2urz89Rq9Vs3LiRP/74A1dX1+tftGzFlaY1XI2CX2ZVWXHVyqEVS0csxd7SnkjzSEa8MwJHR0f2REQwz8oSxcWF4nPnuPL00+gLC+uyy0KIZkaCHCFEjVzsrFjygCE/Z93xRH6qIT/HxcUFc3Nz4+fnzp2r+aL27jD9R8OKq+jd8Mf/VVpxBYbyD4uHLcbKzIqTVieZtHASarWaTdu3c+WJxzFzcKDw8BHiX3gBpaSkhm8khLjTSZAjhLiuXr4u/GNUeX7O+eTq83MAtFotTz75JF27dmXPnj01X7RFJ5j8DaCCw8vg4NdVmvT27M2/7vkXKlQctjvMzIUz+emnnwh++GF8ln6Bytqa/F27SXxlHopOd6vdFEI0QxLkCCFu6Ml72jKkozvFWj3P/HCU/Bryc8zNzcnJyUGr1XLfffeRmJhY80U7joERbxpe//kKXN5epcmINiN4Peh1AMJdwintbJjasu3ZE/dF/wELC3I2bSL5nXekcrkQogoJcoQQN2RmpuLD+7vh6WhNVFo+r/92utqgQqVS8c0339C5c2dSUlK47777KLnedNKA5+HuB0DRwc8PQ3pklSb3d7yfZ7o9A8DCAwsJiwkjLi6OQc89x7bhw0ClImvVj6R98klddVcI0UxIkCOEMIlLhf1z1h5L4JfD8dW2s7OzY+3atWg0Gvbv38+LL75Y80VVKpjwCfj0Nay4qqbGFcBTXZ9iasepKCjM2zOPfy39F+fOneP5Tz/l8NgxAGQs/ZKMZd/WSV+FEM2DBDlCCJP19nXhxZEdAJj/+2kuJFe/jLt9+/asWLECgM8++4z//e9/NV/UQg1TfwCND1y9DL88XGXFlUqlYl6feYxsM5JSfSmHOx1m8oOT0ev1PPbpp1wYNxaA1H//m6zVq1F0OvIPHCR7w0byDxyUnB0h7lAq5Q6eyM7JyUGj0ZCdnY2jo2ND344QTYJer/Dw8kPsvphGO3c7fp8zEDu1RbVt33zzTd566y1sbW2JiYnB3d295gsnn4b/joLSfOj9GIz7oEqTEl0JT299moPJB3G2dMZ2lS1hG8JwcHBgzezZtArbDCoVZhoN+gqFQy08PWnx6jwcR4261e4LIRoBU/9+y0iOEOKmmJmp+Oj+u2nhqOZyWj5vrDtdY9v58+czY8YMVq9eff0AB8CzM0z+GlDBoW+qXXFVVv4hwCWAzNJM9NP1DLhnALm5uTy4ahXJgYGgKJUCHABtSgoJz79AzubNteixEKKpkpEcGckRolYORGXwwNcR6BX4z5Su3NfLp24uvOdD2PYWqMxhxhpoN7RKk/TCdB7a9BDxefG0s25H7L9iOX7sOEFOTixr4Vn9dVUqLFq0oP22ragq7OkjhGh6ZCRHCFGv+rZ15aVr++e8se40F1NuXGYhOjqapUuXXr/RwL9D12mGFVe/zIL0S1WauNm48dXIr3CxduFy0WUCXg4gZMhg/uXqVvN1FQVtcjIFh4/c8D6FEM2DBDlCiFp7enA7Bvm7UVRq2D+noKT6/XMAUlJS6NmzJ08//TS//fZbzRctW3HVqjcUXatxVZhZpZmPow9LRyzFztKO00WnGTXWE3eL8tygmgaps9b9hjaz6vWEEM2PBDlCiFozM1Px0dRueDiouZSax/x1Z2ps26JFC2bOnAnAzJkzuXDhQs0XtrSGaSvBsRVkXKp2xRXAXa538cnQT7A0s2R34Snj13/PzmZOQgIl1QQ6Ob+uJfKewcQ/+yw5W7agl7IQQjRbEuQIIW6Jm72axQ90x0wFq4/Es/pI9fvnAPznP/9h0KBB5ObmMnHiRHKvV0nc3sNQ48rSDqJ2wp/zqm3Wt2VfFg5ayHkfM9IdIF2r5a2UZHbk5zEjLpZVmZlc1RpGmMwcHLAKCIDSUnK3bCXh2ee4NOgekt56i8Ljx2XXZCGaGQlyhBC3LKitK38fYdg/543fThNZQ36OpaUlP//8M15eXpw7d47Zs2dfP7Dw7AKTvjK8PvS1YdVVNYJ9g5nX7zWWjzTD1cKCj7y9sVKpOF1UxNupKQy+fIkn46+we9hQPL7/H37r1uHy6CNYuLujy84ma9WPxEx7gKjRY0j7/HNK4msO1IQQTYesrpLVVULUCZ1e4eFvD7InMp0OLexZ97eB2FhVv4opPDycwYMHU1payr///W/mzp17/Yvv+QC2/dOw4uqhX6HtkGqbvbjzRbI3h/HwFj36zFL+yMllY24Op4uKjG0+++wznnnGUCZC0enIj4gge906crdsRSksNLaz6dUTTWgojqNHY+7gcHM/DCFEvTL177cEORLkCFFn0nKLGbt4D2m5xdzfqxX/nnJ3jW2/+OILnnnmGYKCgtizZw8WFtVvKAiAosDaJ+HkT2DtBI9vB9d2lZro9DqC1wSTUpCCSq9w1xUF5zzItIfjliVkHcwh/0g+lw5dwvPaMvPvv/+eXbt2MX36dAb16kX+9u3k/P47+eERhu8JqKyssB8+DE1oKPYDBqCytLzln5MQ4tZIkGMCCXKEqHv7L6cz45sD6BX44L67mdyzVbXtFEXhu+++Y9q0aVhbW9/4wqVF8N14iD8Erv7w2FawcTIePpR8iEfCHrnhZZYFL6O3Z28A7rnnHvbs2QNAy5YtmTp1KtOnT+fuVq3I2bCB7HXrKLl02XiuuYsLjuPHoQkJxbpTICqV6sb3LYSoc7JPjhCiQfRv58bzww35Oa//dppLqdXn56hUKh5++OFKAc51/81laW2oceXYCjIiYfVs0JUvWU8rSDPp/j4//jn7EvZRqi/l7bff5oknnsDZ2ZmkpCQ+/vhj+vTpQ6d77uHzpCT8fv8d3zWrcZk1E3NXV3RXr5L5v++JmTKFqAkTSP/qa0qTkkz6vkKI209GcmQkR4g6p9MrzFx2gH2XMujYwoHf/jagxvwcAJ1Ox4IFC0hJSeGrr766/ghJ0klYFgylBdDnSRj7b8D0kZwyGrWG4a2HE9wmmLtd72bH1h2sXLmSdevWUVBQwJgxY9i0aZOxfUpCAnaXLhnyd7ZuQylbeq5SYRvUF01IKA4jR2Jub2fyPQghakemq0wgQY4Q9adifs7UXj78a0rXGtseOnSIoKAg9Ho9X375JU888cT1L372d/j5IcPr8R9Br0eMOTmpBakoVP21pkKFs7Uzw1sPZ1vcNq4WXTUec1I7GQIe32Dusr+LTRs20bJlS4YONZSUiIuLw8/Pj4EDBzJ9+nQmBgdjdfgw2b+to+Dw4fLvYWODw8gRaEJCsesXJOUjhKgnEuSYQIIcIerX/kvpPPjfAygKfDT1biZ2rz4/B+D9999n3rx5WFpasmfPHvr27Xv9i+/+D2x/B8wsYMav0HYwW2O38uLOFwEqBToqDCNDHw75kBFtRqDVazmScoSwmLAqAY+z2pkRbUYQ7BtMzxY9sTCzYMWKFTz00EPGNhYWFgQHBzN9+nRG9+yFfvs2stf9TklMTHkbDw8cx49HExqKdccON/NjE0LcgAQ5JpAgR4j69/HWi3y8NRJbK3N+nzOQ9h721bZTFIXJkyezdu1avL29OXLkCC1atKj5wooCvz4Op36ptOJqa+xW3j/4PikFKcamnraevNznZUa0GVHlMlq9lsMphwmLCWNr7FayirOMx1ysXRjR2hDweJR4sPqX1axcuZJjx44Z29ja2vLnn38ycOBAik6eJHvdOnI2bkKXnW1sow4IQBMaimb8OCxuVI1dCHFDTSbIWblyJYsWLeLcuXPY2NgwbNgwFi5ciL+/f43nvPLKK+zatYuoqCiys7Px8vJi3LhxvPHGG3h4eJj8vSXIEaL+6fQKD/33APsvZxDgacjPsbasfhonJyeHvn37cv78eQYPHszWrVuvv7S8tBCWj4OEI5VWXOn0Oo6mHiWtIA13W3d6ePTA3OzGU0el+lIOJR9ic8xmtsZtJbu4PFBxtXY1jvDYXrXl559+ZuXKlSQkJJCSkoLDtb10wsLCsDIzo4eikLt+A7k7d0LptZIUZmbYDRiAJjQUh+HDMLOxMfnnKIQo1ySCnK+++oonn3wSAD8/PzIyMsjJycHd3Z3jx4/j5eVV7XkqlQqVSkW7du0oKCggMTERgM6dO3PixAnMzExbNCZBjhC3R2puEWM/2Ut6XjEP9PFh4aSa83POnz9Pnz59yM3N5cUXX+SDDz64/sVzk+HrYZCTAO2GwfRfwPw6gZGJSvWlHEw6aJzSyinJMR5zs3FjZJuRjGozCqd8J9q3a2881q1bN06cOIG3tzfTpk3j/vHjaZ+YSM7v6yk8ftzYzszODofgYDQhIdj26Y3KxN9bQogmEOQUFxfj7e1NRkYGkydPZvXq1SQmJhIQEEBubi5z5sxhyZIl1Z77+uuv8/zzz+Pu7o5Op2Pq1KmsWbMGgKNHj9K9e3eT7kGCHCFun32X0plxLT/nk2ndCO3mXWPbtWvXMmPGDL788ktmzJhx44snnYBlow0rrvo+BcHvQex+yEsB+xbQpj+YMJJTk1JdKRFJEYTFhLH9ynZyS8qXxXvYeDDSdyTBvsEEOAbw7JxnWb16NdkVpqs6dOjA9OnTmTxoEC1OnyH7998prVA6wsKrJZoJIWhCQ1C3bVvr+xTiTtHog5x9+/YxcOBAwDBl9cADDwAwatQotmzZgr+/PxcvXjTpWh9++CEvvfQSAGfOnCEwMLDadsXFxRQXFxs/z8nJwcfHR4IcIW6TD7dcZPG2SOyszPn92YG0c68+PwcgOTkZT09P0y9eccWVtRMUZZUfc/SC0f+CwJBa3XdFpbpSwpPCCYsJY0fcDnJLKwQ8th6MajOKoS2HEn84nh9X/cj69espulZWYurUqfz4448oikLh0aNk/7aOnD//RF+hUKl1ly5oQkJwHDcWCxeXW75fIZqjRr8Z4JUrV4yvK+bRlCUaxsXFmXSd3Nxcli1bBkD//v1rDHAAFi5ciEajMX74+PjU5taFELX0/HB/gtq6kF+i428/HKWoVFdj24oBTmpqKllZWde/eGAIdJ5ieF0xwAHISYKfZxoCoVtkaW7JPa3u4d2B77Jz6k4+HfYpE9pOwN7SntSCVFacW8Gj2x/lc+3n9Jrbi21nt/Hdd98xZswY46iUSqUixsaG6Xv3sHP2w9i8/U/shwwBCwuKTp0i5d13ibxnMFeefoacP8PQV/jHmRDCdA02krNq1SqmT58OwNatWxk+fDgAM2bM4IcffsDa2prCCsXyqpOWlsaECRM4cOAAAQEBbN++nZYtW9bYXkZyhGh4qTlFjF28h/S8Eqb3bc17E7tct/2hQ4eYNGkS3bp1Y926dTXn3Ol18HFnyEms4Uoqw4jOC6duaeqqJsW6YvYn7Ccs1jDCU6AtMB5radeSUW1GEewbTGe3zqhUKt544w3eeecdwFCdfcyYMUydMIF7FCgNC6Po9Gnj+WaOjjiOHo3m3lBsuneXchLijtekpqt++OEHY8Bj6nTVhQsXGDt2LFFRUQQFBbF+/Xrc3Nxu6h4kJ0eIhrEnMo2Zyw6iKLD4ge6E3F39IgMw5Nn179+f4uJi3nrrLebPn199w+g9htpWNzJrA/gNquWdm6ZYV8y+hH2ExYSx88rOSgGPl50Xwb7BdDHvwuE/D/Pjjz9y8uRJ43E7OzvuvfdeFj77LOq9+8hevx5tcrLxuKWPD5oQQ/6OVevW9doPIRqrRh/klJSU4OXlRUZGBpMmTWLNmjUkJCQQEBBAXl4ezz77LIsXLyYgIACAOXPmMGfOHAB2797NxIkTuXr1KpMnT+b777/HphZLMSXIEaLhfLD5Aku2X8LOypwNzw3Cz63mcgjLly9n9uzZqFQqNmzYwNixY6s2OrUa1jx642/s0xd6PwbthoOd6y30wDRF2qLygCd+J4Xa8hFqb3tvgn2D8Sv04+Cmg6xatYqYmBicnJxITk5GrVaj6PVEbtiA7f5w8jdvRl9QHjDZdO+OJjQUxzGjMddo6r0vQjQWjT7IgZqXkLu5uXHixAm8vLyMw7ILFizgzTffBECtVlNSUoJKpaJ3796Vhm7feOMNxo0bZ9L3lyBHiIaj1el58JsDHIi+yl0tHVn7TP8a988BeOaZZ/jiiy9wcnLi8OHDtGvXrnIDU0dyjFTQqhe0Hwn+I6FlN6jnZdyF2kL2JuwlLCaM3fG7KwU8rexbMarNKFpmtESfoTeObiuKQvv27SktLWXalCmE+Prhc+IEBeHhoNcbemJpif3QoWhCQ7AfNAiVlVW99kOIhtYkghwwTFWVbQZobW3N8OHDef/9942bAVYX5FxvPvrbb7/l4YcfNul7S5AjRMNKySli7Cd7yMgv4cG+rXn3Ovk5JSUlDB48mIiICLp27Up4eDi2trblDYw5OUlQTe0qUIGdG3SfAZe2QvKpyoft3MsDnnbDwMapLrpYo4LSAvYk7CEsJow98Xso0hUZj7V2aE2wbzDBvsFYZlnSrVs3ciuswLrrrruYGhLCOCdnXMLDKb5wwXjM3MkJx3Hj0ISGYN2li+TviGapyQQ5DUmCHCEa3u6Lacz61pCfs+SB7ky4Tn5OQkICPXr0IDU1lblz5/Lvf/+7coOzvxtWUQGVA51rf+jv/1/5MvKcREOwE7kZLu+ECnvfoDI3TGv5jwT/UdCiE9RjsFBQWsDuhN1sjtnM7vjdFOvKF0j4OvoyxHMIlhcs2bV+Fxs3bqy0gOLtt9/mH1OmkL3ud7I3rEeXlm48ZuXnZygnMWE8lt4170skRFMjQY4JJMgRonFYFHaBT3dcwl5twYZnB+J7nfyc3bt3s2TJEr755hs01eWhnP0d/ny58iorR28Y/X7N++RoS+BKhCHgidwCaecrH3fwKg942g4GtUMtemmagtICdsXvMo7wlOhLjMd8HX25x/Ue9Kf0bFu3jW3btrF161ZjtfTTJ06w58cfGZybh9nevShF5aNDtn36oAkNwSE4GHP7mvcnEqIpkCDHBBLkCNE4aHV6pn99gIMxV+nk5ciap6+fn3NDet2t7XicGQuXthgCnqhdUCF3BjNLw/X8Rxk+3PzrbZQnvzSfnVd2EhYTxt6EvZTqS43H2mraEmQfxOTuk+ngaqhy/vzzz7N48WKsrKwYGxzMvR07EpSUjP7IEUNBU0ClVuMwfDiae0Ox698f1fVqgwnRSEmQYwIJcoRoPJKzDfvnXM0v4aGgNrx9b+cbnqMoCkuXLiUkJATv+pqOKS2C2L2GgOdiGGRGVz7u1KY84PEdCFa21V/nFuWV5LEz3hDw7EvYVyngae/UnlG+o0jbmsZPy37i7NmzxmMODg6EBgczvmVLul+6jC4qynjM3M0NzbhxaO4NRR0QIPk7osmQIMcEEuQI0bjsvJDKw98eAuCz6T0Y17XmzT0B5s+fz9tvv02/fv3YuXMnVrdjVVHG5WvTWpshZi/oyqeTsLAG30HXgp6R4OJXL7eQW5JrHOHZl7gPrV5rPNbeqT2dSzqTtj+NP379w7h7vJeXF7GxsZSeO0/277+Ts2EDusxM43lqf38094biOH48ltd2nheisZIgxwQS5AjR+Pz7z/N8vvOySfk5ly9fplevXmRlZfHMM8/w2Wef3cY7BUryIXp3eS5P9pXKx139ywOeNv3BQl3nt5BTksOOuB2ExYQRnhiOVikPePw1/rTPbk/y3mQ6tOnA66+/DoBWq2XQwIEMaNeO0SozWp08iVJyLVhTqbDr1w/NvaE4jBiBmW39jEwJcSskyDGBBDlCND5anZ4Hvo7gUEwmnb0N+Tlqi5rzaTZt2sT48eNRFOWmtpCoc4oCaRfKR3niwqHCCAuWdtB2CPiPMCxVd6r72nnZxdlsj9tOWGwYBxIPVAp4AlwCCPYNZlSbUVw4cIFRo0YZj3Xp1Il7u3RhVEEhrhWWo6tsbXEcORJNaAi2ffuiMq/7chhC1IYEOSaQIEeIxikpu5Cxn+whs6CUWf3a8Fbo9fNz/vnPf7JgwQLUajX79++nR48et+lOr6MoB6J2lo/y5CVXPu4RWL5iy6cvmFvW6bfPKspi+5XtbI7ZTERSBDqlvBiqv50/LWJbEL0zmh2bd1BSUj7l1q9nT+bdcw93nTlDaWx5oWSLFi3QhExAExKC+to+ZkI0FAlyTCBBjhCN144Lqcy+lp/z+YM9GNul5vwcvV5PaGgoGzZsoE2bNhw+fPima9nVK0UxbD5YFvDEHwRFX35c7QjthhoCnvYjwMGz5mvVQmZRpmGEJyaMg8kHKwc8an80FzVc2nGJfbv2oSgKhw4domfPnhQeP86FFSvQ79qFbV6+8RzrwEA0oSE4jhuHRWP6OYs7hgQ5JpAgR4jG7f0/zrN012Uc1BZsfG4QrV1rzg/JysqiV69exMTE8NNPPzF58uTbeKc3qeAqXN5uCHgubYGCjMrHW95dvmLLu2edVk2/WnSVbXHbCIsJ41DyIfQVgq12tMM52pl3XnoHbwfDarVHHnmEVatWEdy7N+Ps7ekdG4dady1IMjfHbuAAnEJDsR82DDNr6zq7TyGuR4IcE0iQI0TjVqrT88BXERyOzaSLt4bVT/e7bn7OqVOnyMjIYMiQIbfvJm+VXg+Jx8pzeRKPVj5u42wY3fEfVedFRTMKM9gWt43NMZs5lFI54Onq1pWRbUayZPYSjh4uvyeNoyPjundnNNA9KRnza8vOzeztcRgdjFNoKDY9e6Kq5zpg4s4mQY4JJMgRovFLzCpk7OI9ZBWU8nB/X94M6WTyud9//z1dunShc+fOWDSVTe/yUuHStmvlJrZBUXaFg9eKipat2PK8u86KiqYXprMtdhthsWEcTj6Mcq0shqIotM5tjeqYiuObj5OYUL6T9IDevfn1wQfJWfc7pYnlX7f08sIxNMSQv+NXP8voxZ1NghwTSJAjRNOw/XwKjyw/DMDSGT0Y3fn6++cApKWl4eHhAYCtrS29e/cmKCjI+OHpWbd5L/VCp4X4Q+W5PCl/LSrqcS15eSS0HVpnRUXTC9PZEruFsJgwjqYcLQ949Aread5oj2o5tuUYzz37HG+++SaKXs/Vfft477XXGJZxlba68pwf67u7ogkNxXHMGCycnevk/oSQIMcEEuQI0XQs/OMcX+6KwsHago3PXj8/B+DixYvMmTOHAwcOkJOTU+X4yy+/zPvvvw+ATqdDq9WiVtf9PjZ1qgGKiqYWpLIldgubYzZzLPWYMeDRa/Xc7XQ34wLHMbLNSPb+udeYB9XFz4/x7u4Mz8zEqyyfyNIS+8H3oAkJwX7IEMxux8aNotmSIMcEEuQI0XSU6vRM+yqCI7GZ3N1Kwy9P9cfK4sZTNXq9nvPnzxMREWH8OH36NMuWLTPuqXPw4EEGDRpEjx49Ko32tG7duvGWOmiAoqIp+SlsjdtKWEwYx1KPGb+uQoV3mjepm1I5tfcUpaXlJSeCOnRgrI0No/ILcLy2z46ZRoPj2DFoQkKw6dat8f6MRaMlQY4JJMgRomlJyCpk3LX8nNkDfFkwwfT8nIpycnIwMzPD/lo17s8//5y//e1vVdp5enoSFBTEK6+8Qt++fW/p3uvdbS4qmpyfbJzSOpF2wvh1XZ4O50hnssKzOHfoHGV/Yk788Qeep0+TvX4DpcnJxsDGsk1rNCGG/B0rn7rfIFE0TxLkmECCHCGanm3nUnj0O0N+zpcP9SS4063n1iiKQlRUVKXRnuPHj6PVGnYM3r17N4MGDQIgLCyMdevWGUd7/P39G99IxG0uKpqUl8Tm2M1sjtnMyfSTxq9rM7U4nnXEOtWaX1f9iquNK4pOx4Pjx5MfE0NwSQkDzC2wupY8bdOzp2H/ndGjMZffyeI6JMgxgQQ5QjRN7206x1e7o3C0Nuyf4+NS9/WVCgsLOXr0KBERETz99NPYXqvh9Pzzz7N48WJjOxcXl0pTXIMGDcK6se0XcxuLiibkJbAlxjDCczrjtPHrZiozenv2ZpDbIJ7s/yTFxcUAaOzsGOPtTXBhEb1sbDBXqVBZWWE/bBia0BDsBw5EZVm3u0GLpk+CHBNIkCNE01Sq03P/l+Eci8vibh8nfnmyn0n5OXVh+/btbNq0iYiICA4fPmz8Y10mPj4eb2/DRnrHjh3DwsKCwMBAzBtL3afbWFQ0PjfeOKV1JuMMYBg1K4ktwfqkNQl7E7iaetXY3tPJiSe8WzFNW15zy9zFBcdx49CEhGDduVPjGzUTDUKCHBNIkCNE0xWfWcC4xXvJLizl0YF+vDE+8LbfQ0lJCSdOnDBOccXGxrJ3717j8fHjx7Nx40bs7e3p06cPQUFB9OvXj759++Lu7n7b77cKRTEkLJcFPPVYVPRK7hU2x2wmLCaMc1fPGb69XqHoYhGWJyyJ3x9PXnYeixYt4m9jx5L92zoS1v1GcmoqflaGQMuqXTs0oaFoJozHsuWNtxEQzZcEOSaQIEeIpm3r2RQe+58hP+erh3oyqg7yc+rS/fffzx9//EFeXl6VY126dOHEiRPGkQlFURp+lOI2FRWNy4ljc6wh4Dl/1bAqTK/VU3i6kEH9BnFv93sZ1noYv65YzSOPPUYXDw9Gm5szxsYWT0tLUKmw7dsXTWgoDiNHYm5vd4sdF02NBDkmkCBHiKbvnQ1n+WZvdL3m59wKnU7H2bNniYiIIDw8nIiICM6dO8c999zDrl27jO169uyJra1tpfyesmmvBnGbiorG5sQaR3guZF4wft3CzALzMHOO/XgMvc7wfVUqFX08PBiDilEODjiZm6OytsZh5Eg0ISHY9e+HqrFMC4p6JUGOCSTIEaLpK9Ea8nOOX8mim48TP9/G/JzaysrKIi0tDX9/f+PnztXsBtyqVSuCgoKYMGECM2fOvN23WdltKCoanR1tCHhiw4jMjARAm6Ml70geuqM6kk4lGduqLSzYP3gINvHxxq9ZuLvjOH48mntDse7YsXb9FE2CBDkmkCBHiObhytUCxi3eQ06RlscH+fHKmLs4GH2V1NwiPBys6ePngrlZ401Y1ev1REZGVlrCfvLkSfR6wwjG7NmzWbZsGQBarZZ//OMfxhwfPz+/2z/NpddB4vF6LSoalRVFWGwYm2M2cynrEgAlGSXkHcyj5EgJXm5e7N25F4sL0WSv+51FX39FO71Cfzs7rFQq1AEBaEJCcBw/Dstr5T1E8yFBjgkkyBGi+dh8Jpknvj8CgLOtJZkF5bvuttRYs2BCoEk1rxqLvLw8jhw5QkREBN27d2fUqFGAYcVWjx49jO3c3d0rTXH17t0bB4db39345m62fouKXsq8ZMzhicqOAkBfrMfaxpoB3gPoY9+HWf1modfrcbK2JtjGlrH29vS0scHM3By7/v0N+TsjhmNmY1OHHRcNRYIcE0iQI0Tz8sjyQ2w/n1rl62XjHF+YWNyzMYuMjOSzzz4jIiKCo0ePViqhAPD222/z+uuvA4bfcQkJCXTs2BGzOqpWfkP1WFRUURQuZRkCnj+j/yQmJwaA0sxSMv/MJO9gHgWZBcb2LW1sGW1tzWSNhrZqNWa2tjgEB6MJDcW2T29Ut+tnIuqcBDkmkCBHiOZDp1cY8P52knOKqj2uAjw11ux9eVijnrq6GUVFRRw/frzSNNc333zDiBEjAFizZg1TpkxBo9HQt29f+vXrR1BQEH369MHFxeX23GROoiHYidxsWLlVUmGlmcocWgeVr9jyCDS53ISiKERmRRIWE0ZYTBixObEoOoX88/nkHsgl90guJfmGTQ8/mDaNkLR0SuPjjavYLFq2RDNhAprQENTt2tVDx0V9kiDHBBLkCNF8hF/O4IGvI27Y7ofH+jKgvdttuKOGUXEp+meffcbcuXMpLCys0q5jx44sX76coKCg23dz2hLDXjyRmw3V1P9aVNTRuzzg8RsManuTLqsoChczLxoDnrjcOPQlevJO5ZF7IJdpr03j3i6h9E5z5LP5/+TnvXsYZ2vHGEdH3C0ssO7cGU1oKI7jxmJxu4I/cUskyDGBBDlCNB/rjifw/I/Hb9jOXKWirbsdfm52+Lnb0dbNjrbu9vi52eFqZ9Xwe9XUsdLSUk6fPm1cvh4REUFkpGHlUlRUFH5+hjIOS5YsYc2aNZXyezw963nfoRsVFfUdcG2J+kiTi4oqisL5q+eNU1rxeeWrr2wsbEh4P4HE04mAYXSvj50d4+0dGOHggEatxn7QIEM5iaFDMVPXfrdnUb8kyDGBBDlCNB+mjuRcj4O1BW3d7Wnrdi0IcrMzBkS2VhZ1dKcNLz09ncOHDxMcHGwM6iZNmsTatWsrtfP19TUGPI899hh2dvW46V49FBVVFIVzV88ZR3gS8hLQ5mjJPphN7oFc8iLLp84sVSqG2dvzYUsvVCoVZg4OOI4ejebeUGx69Gh2wW9TJ0GOCSTIEaL50OkVBv5rO8nZRVT3S60sJ2fV40HEXS0gOj2f6PR8LqflEZ2eT0JWIdf7bejpaF0p6GnrbkdbN3taOdtgYd70E1gvXLjAvn37jJsWnjlzhrI/D1ZWVuTk5KC+NrLxyy+/oCgKQUFB+Pj41E8AUMdFRRVF4WzGWWPAk5ifSElaCdkHssmJyKEwvpABA3vwy8QHyFu/CW1SEkcKCuhqY4Nt69bG/B2rNm3qvq/ipkmQYwIJcoRoXv48ncTTKwx7tlT8xWbK6qqiUh1xVwuISssjKj2f6LR8w3/T87maX1LtOQAWZipau9pWmvbyczNMg7k7qJvsCEBOTg6HDh0iIiKCzMxMFi1aZDzWrVs3Tpw4AUDLli0rTXH17Nmz7kd86rioqKIonE4/bQh4YsNIzk+m6EoRiqLg1s6Nod5D6B7pwNTHX8fZwoLR9vaMdXCku40Ndt27o7k3FMfRozF3cqrbfgqTSZBjAglyhGh+/jydxFvrz5KUXb7K6lb3yckqKCE6PZ+otHzjCJAhAMqjqFRf43n2aosq015t3ezxc7fDXt00p78UReGFF15g3759nDhxAm2FiuEAHTp04MKF8vIMiYmJtGzZsu6CPZOLil5bpq5pdcP+nEw/aSwtkVKQAkDuqVwSvjZMb5VpaWHJOEcHxjo4EmBvj8PQoWjuDcV+0CBUVlZ10z9hEglyTCBBjhDNk06v3JYdj/V6heScomsBUJ5x5Cc6PZ8rVwvQX+e3q4eDutrgx8fZttGXpShTUFDA0aNHjQnN4eHhDB06lBUrVgCGnZydnZ2xsLCoNNrTp08fNBpN3dxEUfZfioqmVD7u0alCUdE+1y0qqlf0nEw7SVhMGJtjN5OSm0Le2TyyI7LJPZKLrkhnbPuZtzdD7Q2bLpo7OeE4dqyhnESXLk129K4pkSDHBBLkCCHqS7FWx5WrBcbRH+N/0/NJzyuu8TxzMxU+zjaVp76u5f+0cGz8019FRUVYW1sDhtVbgYGBFBdX7q9KpeKuu+7iscce4+9//3vdfXNFgeSTFYqKHvpLUVHNX4qKtqjxUnpFz4m0E4aAJ2YzKdkp5J7IJTsim4ILBfzz00cJvqzGfscx1l66RI5ex2gHR7zat0dzbyiaCROwbMgCq82cBDkmkCBHCNEQsgtLiakw7RV1Lfk5Oj2fghJdjefZWpnj61oW9BiWwPu5GYIhjU3NIxQNqaSkhBMnTlTasDAqylCaYf78+bz11lsApKamMn36dONoT9++fXF3d7+1b15HRUX1ip5jqccIiwljS+wWUnNSMbM0jLY5mtsT9X/nSU7KwgwIsrVlnKMjI+wdaBEUhCY0BIfgYMxvd6mNZk6CHBNIkCOEaEwURSE1t9i44qti8nPc1QJ015n/crO3qjTtVZb83NrVFrXFzVcEr0+pqakcOHAAf39/AgICAFi/fj0hISGV2rVv394Y9IwdO9a4p0+t3LCoqAu0H37DoqI6va5SwJNekM7VbVfJCs+iMKp8nx8rlYrBdnbcq9EwzNUNh+HD0YSGYDdgACqLppmP1ZhIkGMCCXKEEE1FqU5fefrrWuJzVFo+qbk1T3+ZqaCVs22Faa9roz/udrR0tMaskZS4SEhIYOPGjcbRnnPnzlU6vnz5cmbNmgVAdHQ0R48eJSgoCO/aTgnVQVFRnV7H0dSjxoAnKS6J7IhssiOyKU40vCfjWrbgP47OgCGIxc0N1/Hj0ISGor7rrkY//dhYSZBjAglyhBDNQV6xlpgKe/4YV4Cl5ZNXrK3xPGtLM+P0V8VRoLZudjjZNuxqoczMTA4ePGgMehYvXoy/vz8An3zyCS+88AIArVq1IigoyFiXq0ePHsacIJPVQVFRrV7LkZQjhoAnZgtJlwwBj30Xewa2aMl9UR5od13iibPnGe3gyHhHR3p36YLTvaE4TpiAZYua84NEVRLkmECCHCFEc6YoCml5xUSnVcz/MYwAxV0toFRX869/Z1vLSsnP7a7l/7RxtcXasmGnv5YtW8aSJUs4efIken3lJfyWlpYcOHCA7t27A4ZEaLX6JhO2sxMMtbVqWVRUq9dyKPkQYTFhbIvbRlZxFgBpa1JIWZ9mbNfK0pKxDo6M02joNmQwmtBQHEaMwKw+d5ZuJiTIMYEEOUKIO5VWpyc+s7Da5OeKewz9lUoFXhqbClNf5ZsgejnZ3NYK73l5eRw5csRYlys8PJyrV6+SnZ2Nra2h7MNzzz3HTz/9VGkJe69evXAwNRG4YlHRyC2QfqHy8RsUFS3Vl3Io+RCbYzazOWoziccSyQrPIvdoLvri8gCto1rNEi9vfJyccBw5Ak1oKLZ9+6Iyb1z5VI2FBDkmkCBHCCGqKijRlk97XRsFunwtEMotqnn6y8rCDF9X2yrJz23d7XG2taz3/BNFUUhKSsLLy8v4tf79+xMeHl6pnZmZGZ07dyYoKIglS5ZgdTMb+WXGGIKdyC2GXZhvoqhoqb6Ug0kHDcvSIzeTeDCRrIgs8k7mYWNpztZed+OUlg/A8cJC/Fq2pO2UyTiGhGDdoUOtfibNlQQ5JpAgRwghTKcoClfzS/5S9sKQ/BybUUCJrubdnzU2lhWCnvKl735udthY1d9oRVFREceOHau0hD0uLg4AHx8f42uABQsWoFKpjBsWuri4XP/ipYUQu69WRUVLdaVEJEUYAp5zm0mPTceuvS3+iTDynBVv/3ya1JJS+tnZMd7BkTG9euI9eTKa8eOxcHOrs59PUyVBjgkkyBFCiLqh0yskZhVWm/yckFV43XO9NNYVRn7Kk59bOdvWy/RXYmIiBw4cID8/nxkzZgCGAM7NzY2rV68a23Xs2NE4xTVgwAC6dOly/QvXsqhoqa6U8KRwwmLC2BG3g8z0TOI+iqUgpvznplapGGJvzziNE6ODR+ExcSIOw4djdrNJ1s2EBDkmkCBHCCHqX1GpjpiM8tGfsuTnqPR8sgpKazzPytyM1q621Y4Audlb1en0V2lpKV9//bVxtCcyMrLS8cGDB7Nz507j51u2bKFr1660qGlVVHEexOwxBDwXN0NOfOXjNRQVLdGVEJ5oCHg2HdpE0t4kcvZnUZRSHjDNcHLm1RYtMLO3xyF4lCF/p1cvVNUsc2+uJMgxgQQ5QgjRsDLLpr/+kvwcnZ5Psbbm6S8Hawtj4rNfhdEfPzc77Oqg+Gl6enqlJeyDBw/mtddeM9xzZqZxKsvX17fSEvZu3bpVzfGpZVHRYl0x+xP282fMn2zcs5GUvSnkhmcxY4A3D2fY45ql42RhIRtzcwht246BD07HKfRe1G1vYdPEJkKCHBNIkCOEEI2TXq+QmF1YadqrLAcoPrOQ6/3lauGorjTtVbYCrJWzDZbmtz7acebMGaZNm8aZM2f4659QtVrNG2+8YQyIyo5XGnWqRVHRIm0R+xL38WfUn+yM30mxtpCO8ZC8PIG9l7MA8LG0ZLyjIxN79abnQzNwHDsWC2fnW+5vYyRBjgkkyBFCiKanqFRHXIXdn8uSn6PT88nIL6nxPAszFa1dbI2bH5ZNfbVzt8Pd4eaLn+bk5HDo0CHjEvaIiAgyMjL46quvePzxxwE4evQo48ePr7KEvWyJO3q9YfPBmygqWqQtYl/CPsJiwlj35zrStqeQeywHbWn5n/MAtZpxTk48de9EPCZNxH7IEMxuZhVZIydBjgkkyBFCiOYlu6CU6Izyqa+KOUBFpTVPf9lZmRsLnpbn/xg+HKxNK36qKAqXL1/G2dkZV1dD7avPP/+cv/3tb5XamZubc/fddxMUFMSTTz5J165dyw8ai4puNmxIWKWoaLcKRUV7UKgvYW/CXjac28CG9RvI3J1K9tk89HpoaWHBlrbtMFOpMNM4oh45Eo9Jk7Hp3q3Jl5OQIMcEEuQIIcSdQa9XSMktIjrNsOdP9LXAJzo9nyuZhdctfuruoK42+bm1iy1WFtef/iooKODo0aPGkZ7w8HASExONx7du3crw4cMB2Lt3Lzt37iQoKIjevXujcbCHxGMViooeq3xxGxfD6I7/SGg3nAIrG/Yk7OG3E7/xx7o/sM8rZbqdhkGnFTQ5CiMuX+YuazUhfn5Mmj0b7/vuw8rHp/Y/1AYkQY4JJMgRQghRotUTd7WgUvJzWTJ02nWKn5qbqfBxtjFOfRl3gXa3w9PRutrREkVRiI+PNwY9CxYsMP79+cc//sEHH3wAGHJ4AgMDK01z3eXjinn0DsO01g2Kiha4+rM7aS+bYzazJ24Xjjsy2L481tjaWqVimL09E7t3Z8ITT+I2YTzmTejvoAQ5JpAgRwghxPXkFJUSU03yc3RaPvkluhrPs7E0NwQ/1ZS/0NhUP/21Zs0a1qxZQ0REBNHR0VWOR0dH4+vrC0BU5EUc8y7jlnHwhkVFC3z6sivjBKt2r2LHmi3k7LlKTlZ57pKjmRn/bNWKyfdORBMaiv2ggagsTZuiaygS5JhAghwhhBC1oSgKqbnF1SY/x10tQHud6S9XO6tqk59bu9qitjDs/pySksKBAwcq7dIcGRlpHB2aOHEiv/32G+3btzeM9NwdQJCnlq6cxzJud41FRfP8BrKzMJnvN69g/5odZEVkklegZW0bXzpe21gw0soC9YBBDHr6aWy6dG6U+TsS5JhAghwhhBB1rfRa8dNKU19p+USl55GSU/P0l5kKvJ1tqk1+bulojXmF5e9Dhgxh165dVa5hbW3NwAH92fzlG6gubbluUdFcv3vYZqZlxZZfKC2JZMCpUgaeUXj/UiLrc3LwtbRiXOtWPDTzYXo89iiWFWqCNTQJckwgQY4QQojbKb9YWynwKUt+jkrLJ7e45uKnagszY8BTlvzsallKRvQZzhw/YhzxycrKon///uzbt8947tiRQ7FXcglqoSXILo4eHlqsLa6NzphbQZv+5LYdzE57BzanHuXPl38i9mQWpRVGozpZWzP+rgBmPv03Oj7wAOb2dvX2MzKFBDkmkCBHCCFEY6AoCul5JdUmP8dm5FOqq/lPtbOtofipr4st9sVpuFrpCB46AF9XO0qLCnByckKvL18+b2lpQXc/d4I8SwhuVcBY/wr5N86+5LQbyia1hmWbdnLxt6Ncic2jLN7xt1Lza0d/rIYOwmvKA9j164fKouoO09rSEk5u/ZHcpDgcWram64hpWFjW3T49EuSYQIIcIYQQjZ1Wpychq7DStFf0tdeJ2UU1nqdSgaedBfbZl9GnXORq9Blizx0nMyPd2GbGfaF8//dRELkZffRe/rMnjz7e5vT2Nsfe1oZs3wGstXJnxabDXAw7zwgrW+baGKqgF+r1vJyRyvjhQ3lo3hu4dO8OwN4fFmH2ybc455QHVpmOZuifn83AB/9RJz8TCXJMIEGOEEKIpqywRFeh3leecQVYVFoeOUVVp78URUGbnYI++SKWVy/TsVsfho8Joa2bHaq0C9w/biRgyA/q7GFGv1bmBF378OjYkR1eHTmZlI1TRCz5ezN5I9aw54+NSsVgTzeGdevIqMtpWKpUVExX1gMq4Oobj9ZJoNNkgpyVK1eyaNEizp07h42NDcOGDWPhwoX4+/tf97zFixezdOlSLl++jEajYdy4cSxcuBBPT0+Tv7cEOUIIIZojRVHILCglKi2vSgHUmIwCSqopflqSFkP2/p8oTbpAaXZqleNvD1Xz+j2GaulJZnasNG/Dxi2JnDqURHqFavJO5uaMsnfgMRcXWlUoJaEHsjXm9Nl79JanrppEkPPVV1/x5JNPAuDn50dGRgY5OTm4u7tz/PhxvGrI5H711VdZuHAhAP7+/sTHx1NYWEiHDh04evQodnamJURJkCOEEOJOo9MrJBqnvyqXv0jMNhQ/1eZmUJJ4geLE8xQnXqAkOZLRk6bwdKdCBnKMPedSCP2xkAA3M3q0t0TxcSf5ZCEnL2WRoTPsHxTm1xafauplFX40jx5jZt5SHxp9kFNcXIy3tzcZGRlMnjyZ1atXk5iYSEBAALm5ucyZM4clS5ZUOS85ORkfHx+0Wi0vvfQSixYt4uTJk3Tr1g1FUVi0aBEvvfSSSfcgQY4QQghRrqhUR2xGAdHpeVw27gGUz+XkLDILSlGZW6BCj/rAf7mwc12lc6d0b8H8fCcOFhRwvLCQZ9zcqv0eKXMfYMij82/pPk39+101Jfo2OXz4MBkZhsJjkydPBsDLy4ugoCC2bNlCWFhYtedt27YNrVZb6byuXbvSvn17IiMjCQsLqzHIKS4upri4fI+CnJycOuuPEEII0dRZW5rT0dOBjp4OVY5l5pcQnXEt+Xno+5yd+iwnjxwm+fxBzOOPklmci4VKRX87O/pfZ0Ylvzi/PrtQSYMFOVeuXDG+9vDwML5u0aIFAHFxcTd9XmRkZI3nASxcuJC33nqr1vcshBBC3Kmc7axwtrOiR2vna18JAIaj1ysk5RRxYO2nXP1sGU55UF3ZUj2Q7QDu3m1u2z1fv3xqPapplqzs6zVtI13b8wDmzZtHdna28aNiwCSEEEKIm2dmpsLbyYaOHQJR+uShwhDQVFS2ukrfOw87tzsgyGndurXxdUpKivF1aqoho9unhvLvtT0PQK1W4+joWOlDCCGEELcuoG8w/h6WZAzNI9u+8rFsB8gYmkd7D0sC+gbftntqsCCnd+/euLq6AobKqwAJCQmEh4cDMHr0aAACAgIICAjg008/BWD48OFYXNtdcfXq1QAcP36cS5cuVTpPCCGEELePuYUFif0WMMAjh15jEikMziZ1SD6Fwdn0Gp3IAI8ckvotwLyaHZLrS4MFOVZWVrz33nsA/Prrr7Rt25bAwEDy8vJwc3PjlVdeAeDChQtcuHCB9HTDDo2enp7MnTsXgI8++ogOHTrQv39/FEXB39/fuCRdCCGEELdX9+BZnOi/mExzV3o45zPYM5sezvlcNXflRP/FdA+edVvvp8ESjwGeeOIJ7OzsjJsBWltbM2nSJN5///0a98gBePfdd2nRokWlzQDvv/9+3n//fZP3yBFCCCFE3esePAvd8Ac5cyCMwswEbJy9CegbjOdtHMEp0+A7Hjck2SdHCCGEaHpM/fvdYNNVQgghhBD1SYIcIYQQQjRLEuQIIYQQolmSIEcIIYQQzZIEOUIIIYRoliTIEUIIIUSzJEGOEEIIIZolCXKEEEII0SxJkCOEEEKIZqlByzo0tLLNnnNychr4ToQQQghhqrK/2zcq2nBHBzm5ubkA+Pj4NPCdCCGEEOJm5ebmotFoajx+R9eu0uv1JCYm4uDggEqlqrPr5uTk4OPjw5UrV5ptTazm3kfpX9PX3PvY3PsHzb+P0r/aUxSF3NxcvLy8MDOrOfPmjh7JMTMzo1WrVvV2fUdHx2b5P25Fzb2P0r+mr7n3sbn3D5p/H6V/tXO9EZwykngshBBCiGZJghwhhBBCNEsS5NQDtVrNggULUKvVDX0r9aa591H61/Q19z429/5B8++j9K/+3dGJx0IIIYRovmQkRwghhBDNkgQ5QgghhGiWJMgRQgghRLMkQY4QQgghmiUJcm5g5cqV9OjRAxsbG1xcXJgyZQqRkZE3PG/x4sUEBgaiVqvx8PBg9uzZJCcnV2qTnJzM7Nmz8fDwQK1WExgYyOLFi+urK9WqTf9eeeUV+vXrR4sWLbC2tqZt27Y8++yzpKamVmqnUqmq/Xj99dfrs0tV1KaPDz/8cLX3/tfNI5vie7hz584a3xuVSsXy5cuNbRv6Pdy9ezdjx47F3d3d+L2XLl1q0rlN5RmsbR+bynNY2/41lWewNv1rSs/gBx98wJAhQ2jZsiVqtZo2bdowa9YsoqKibnhuo3gGFVGjL7/8UgEUQPHz81McHR0VQHF3d1cSEhJqPG/evHnG8/z9/RUbGxsFUDp06KDk5eUpiqIoubm5ir+/vwIoNjY2xteA8uqrrzbq/gGKSqVS2rdvr3h5eRmv0blzZ0Wn01VqByjdunVT+vbta/xYunTp7eieoii17+OsWbMUQPH29q507xMmTDC2aarv4ZEjRyr1qW/fvoqvr6/xOn/++aexbUO/hx999JFiYWGhdOjQwXgvX3zxxQ3PayrPoKLUvo9N5Tmsbf+ayjNYm/41pWewTZs2CqC0bt1a8fPzM96Pp6enkp2dXeN5jeUZlCCnBkVFRYqrq6sCKJMnT1YURVESEhIUBwcHBVDmzJlT7XlJSUmKhYWFAigvvfSSoiiKcuLECUWlUimAsmjRIkVRFGXRokXGX1InTpxQFEVRXnzxRQVQLCwslKSkpEbZP0VRlNdee01JTU1VFEVRtFqtMnnyZOP/mEePHjW2K/tadHR0vfalJrfSx7JfsAsWLKixTVN+D/9q3LhxCqB07NhR0ev1xq839HuYnp6uFBQUKNHR0Sb/AWkqz2CZ2vRRUZrOc1jb/jWFZ1BRat+/v2qsz+A777yjxMbGGj9/4YUXjPf066+/VntOY3oGJcipwd69e41v5MqVK41fHzlypDEyrc6KFSuM5+3fv9/49bIIdeTIkYqiKMqIESOMUW2Zffv2Gc/94Ycf6qlnBrXtX3U++OAD47XOnDlj/HrZ11xdXRUbGxslMDBQee+995SioqI67UtNbqWPZb9gNRqNYmVlpbRq1UqZOnWqcunSJWOb5vIenj171vjL56uvvqp0rKHfwzI38wekqTyDf3WrfyQb63NYprZBTmN+Biu6lfevKTyDZdasWWO8p40bN1bbpjE9g5KTU4MrV64YX3t4eBhft2jRAoC4uLhbOq+sXXVtrnf9ulLb/v1Vbm4uy5YtA6B///4EBgZWOu7m5karVq1Qq9WcPXuWV199lZkzZ97q7ZvkVvtobW2Nt7c3rVq1Ij4+np9++onevXuTkJBQ6fpN/T1ctGgRiqLg4eHBQw89VOV4Q76HtdFUnsG61Jifw1vR2J/ButJUnkGtVsunn34KQNu2bRk+fHi17RrTMyhBTg2UGjaCLvu6SqW6pfOqa1fxazVdv67Utn8VpaWlMXLkSM6cOUNAQACrV6+udPzAgQOkpaVx/PhxEhISGDZsGAA///xzpYegvtxKH+fOnUt6ejpnzpzh8uXLxkTCzMxMvv322xqv39Tew+TkZH744QcAnn32WaytrSsdb+j3sDaayjNYVxr7c1hbTeEZrAtN5RnMz89n0qRJ7NixA09PT9avX19juYbG9AxKkFOD1q1bG1+npKQYX5etXPDx8bml88raVdfmetevK7XtX5kLFy4QFBTEgQMHCAoKYs+ePbRs2bJSmz59+hhf29raMnHiROPnt+PhvJU+durUCTs7O+PnDz74oPF12b8umvp7CLBkyRKKi4uxtbXlmWeeqXK8od/D2mgqz2BdaArPYW01hWewLjSFZzA5OZnBgwezfv16OnTowL59+6qMFlbUmJ5BCXJq0Lt3b1xdXQFYs2YNAAkJCYSHhwMwevRoAAICAggICDAO4Q0fPhwLCwsA47+ojh8/zqVLlyqdV/bfS5cucfz4cQB++eUXACwsLGocBmzo/oFhyWT//v2Jiopi8uTJbN++HTc3t0rX3717N6tXr0av1wNQVFTEunXrjMfbtGlTf5275lb6uGDBAtLT042f//jjj8bXvr6+lc5viu8hGP5l9sUXXwDwyCOP4OLiUul4Y3gPTdFUn8Gb0ZSfQ1M01WfQVE35GTxz5gxBQUEcOXKEQYMGER4eTtu2bSu1adTP4C1n9TRjNS3PdXNzMy7PLTtecQVATUvn/P39G9Xy1dr2z8rKypgR36dPn0rLGjds2KAoiqJ8++23CqDY2dkpXbp0UZydnY3Xmj179m3p3630EVDMzMyU9u3bK+3atTO28fT0VFJSUhRFadrvoaIoyieffKIAirm5uXL58uUq124M7+GaNWuUdu3aGZexgmF5fLt27ZTp06critK0n8Fb6WNTeQ5r27+m8gzWtn+K0jSewYpL4/+6jP3rr79WFKVxP4MS5NzAihUrlG7duilqtVrRaDTKpEmTlIsXLxqPV/fm6vV65eOPP1YCAgIUS0tLxc3NTZk1a1aV5XCJiYnKrFmzFDc3N8XS0lIJCAhQPv7449vVNUVRate/sq9V9/Htt98qiqIokZGRylNPPaUEBAQo9vb2ikajUXr27KksXbpUKS0tbfR9fPfdd5UBAwYo7u7uilqtVtq3b6889dRTSnx8fKVrN9X3UKvVGve8uO+++6q9bmN4D8t+yVf3MXjwYEVRmv4zWNs+NpXnsLb9ayrPYG3711SewYrB218/yvrTmJ9B1bUbFEIIIYRoViQnRwghhBDNkgQ5QgghhGiWJMgRQgghRLMkQY4QQgghmiUJcoQQQgjRLEmQI4QQQohmSYIcIYQQQjRLEuQIIYQQolmSIEcI0Szt3LkTlUqFSqUiJiYGgIcffhiVSsWQIUMa9N6EELeHRUPfgBBC1AdHR0f69u0LgFqtbuC7EUI0BAlyhBDNUo8ePYiIiGjo2xBCNCCZrhJC3FZ6vZ5PPvmEzp07Y21tjbOzM/fddx/R0dEALF++3DjNtH37drp164a1tTVdu3Zl165dxuskJyfz4IMP0rJlS6ysrHB3d2fIkCFs3LgRqH66qjqFhYW8+uqrtGvXDisrK1xdXZk4cSKnT582tql4Tzt27KBHjx7Y2NhIICVEIydBjhDitpozZw4vvPACZ86coX379pibm7N69Wr69+9Pampqpbbjx4+npKQEMzMzTp06xbhx40hMTATgmWeeYeXKleTl5dGlSxdsbGzYvXs3hw4duqn7CQkJYeHChURHR9OuXTtKSkr47bff6NevH+fPn6/SfsyYMRQUFKDVajl27BjTpk1Dq9XW/gcihKg3EuQIIW6b6Oholi5dCsB3333H6dOniYmJoVWrViQnJ7NkyZJK7T/66CPOnj3LoUOHsLCwID8/n8WLFwMQGRkJwJIlSzhy5AhxcXEkJCQwbdo0k+9nx44dbN26FYAPP/yQc+fOce7cOezt7cnLy2PhwoVVzvnPf/7D+fPn+eCDDwCIjY3l0qVLN//DEELUOwlyhBC3zeHDh1EUBYBZs2ahUqlwcHAgPj4eoMrUzwMPPABAp06d6NKlCwCnTp0CYMKECQA8+uijtG/fnvHjx7NixQq8vLxMvp+Koz7Tp08HoFWrVgwaNMh4v3/10EMPARAYGGj8WkpKisnfUwhx+0jisRDitikLcAC6detWZdVTmzZtTL7Wu+++y4ABAwgLC+P06dPs3r2bjRs3snPnTmNezs1QqVQmtXNycgLAwqL812fFfgkhGg8ZyRFC3Da9evUyBhMPP/wwERERREREEB4ezqJFi3juuecqtV+1ahUA586dM47glI3o7Nu3j8GDB7N48WK2b9/OZ599BsDu3btNvp/evXsbX//www8AxMfHs2fPHuP9CiGaLglyhBC3Tdu2bXn88ccBeOGFF2jbti1du3bFycmJQYMGcfTo0Urt586dS6dOnejVqxdarRZbW1ueffZZAF555RVcXV1p3749PXv25MknnwSga9euJt/P0KFDGTFiBAAvvvgigYGBBAYGkpeXh729PfPmzauLbgshGogEOUKI2+qLL77go48+okuXLiQmJhIbG4uvry8vvvhilZ2IN23ahFqtRqvV0rlzZ9avX4+3tzcAU6dOpXfv3uTk5HDq1CmcnJyYNm2acfTHVL///jvz5s3Dz8+PyMhILCwsuPfeewkPDycgIKCuui2EaAAqRSaThRCNyPLly5k9ezYguS5CiFsjIzlCCCGEaJYkyBFCCCFEsyTTVUIIIYRolmQkRwghhBDNkgQ5QgghhGiWJMgRQgghRLMkQY4QQgghmiUJcoQQQgjRLEmQI4QQQohmSYIcIYQQQjRLEuQIIYQQoln6f34EImvn+8VUAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots()\n", "for name, g in rdata.groupby(\"model\"):\n", " for fold, gf in g.groupby(\"fold\"):\n", " gf.test_accuracy.plot.line(x=\"epsilon\", label=f\"{name} (folds: {fold})\", ax=ax, marker=\"o\")\n", " g.test_accuracy.mean(\"job_dir\").plot.line(\n", " x=\"epsilon\", label=f\"{name} (average)\", ax=ax, linestyle=\"--\", color=\"k\"\n", " )\n", "\n", "plt.legend()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's it! Workflows have the potential to help scale experiments while easily allowing one to load and evalulate results." ] } ], "metadata": { "kernelspec": { "display_name": "rai_toolbox", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.9" }, "vscode": { "interpreter": { "hash": "ce476f78554e2a0d18ada47e92c39afa3a44848807a61d4717c150e9ec754405" } } }, "nbformat": 4, "nbformat_minor": 2 }