main
/
TDA
7
0
Форкнуть 0
Код Задачи Запросы на слияние Пакеты Проекты Релизы Вики Активность
Вы не можете выбрать более 25 тем Темы должны начинаться с буквы или цифры, могут содержать дефисы(-) и должны содержать не более 35 символов.
Ветки Теги
${ item.name }
Создать тег ${ searchTerm }
Создать ветку ${ searchTerm }
из 'ade1f01236'
${ noResults }
TDA/labs/OATD_LR4.ipynb

2425 строки
207 KiB
Plaintext
Исходник Вина История Escape

This file contains invisible Unicode characters!

This file contains invisible Unicode characters that may be processed differently from what appears below. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to reveal hidden characters.

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "vOpSJBZfFKuo"
},
"source": [
"![](https://camo.githubusercontent.com/518a06d7ca808cd4ad8d5b6deb4ef15983d4649737618153432479f977935bba/68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f79616e646578646174617363686f6f6c2f6e6c705f636f757273652f6d61737465722f7265736f75726365732f657870616e64696e675f6d696e645f6c6d5f6b6e5f332e706e67)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ha8cW_h-FIND"
},
"source": [
"reference:\n",
"\n",
" - [nlp for you](https://lena-voita.github.io/nlp_course/language_modeling.html)\n",
"\n",
" - [YSDA Natural Language Processing course](https://github.com/yandexdataschool/nlp_course/tree/2023/week03_lm)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "H2QPNriQIvTw"
},
"source": [
"# Лабораторная работа №4. Использование нейронных сетей для генерации текста\n",
"В ходе работы мы будем обучать нейронные сети генерировать тексты, похожие на стихи поэтов.\n",
"\n",
"## Цель работы\n",
"Получить практические навыки решения задачи генерации текста.\n",
"\n",
"## Указания\n",
"1. Для работы рекомендуется использовать Google Colab и среду с GPU для ускорения расчетов. Для установки среды, использующей GPU в Google Colab нужно выбрать пункт меню \"Среда выполнения\" -> \"Сменить среду выполнения\" -> выбрать аппаратный ускоритель \"GPU\".\n",
"\n",
"2. Выполнять работу следует последовательно запуская ячейки, анализируя код и приведенные комментарии и разъяснения.\n",
"\n",
"3. В ходе работы будут встречаться вопросы, на которые нужно ответить, создав после него новую ячейку. Вопросы отмечены заголовками 3-го уровня.\n",
"Для ответа досточно 1-2 предложений. Но будьте готовы более подробно его пояснить при устной беседе.\n",
"\n",
"4. Обращайте внимание на комментарии `<your choice here>` - здесь вам нужно будет вставить значения параметров либо исходя из анализа кода\\выборки (где указано), либо попробовать разные варианты. Парамеры, приведенные тут по умолчанию, не обязательно правильные.\n",
"\n",
"## Варианты заданий\n",
"\n",
"Четные номера по журналу - Пушкин, нечетные - Маяковский.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "96PukQZbFNwL"
},
"source": [
"# Загрузка библиотек"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "nyb7MBAsFNhh"
},
"outputs": [],
"source": [
"import copy\n",
"import torch\n",
"import numpy as np\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import matplotlib.pyplot as plt\n",
"\n",
"\n",
"from random import sample\n",
"from IPython.display import clear_output\n",
"from torch.utils.data import DataLoader, TensorDataset"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eyO1tDFiGwFu"
},
"source": [
"При использовании Google Colab следует выбрать среду выполнения с аппаратным ускорителем GPU, что существенно ускорит расчеты. Для установки среды, использующей GPU в Google Colab нужно выбрать пункт меню \"Среда выполнения\" -> \"Сменить среду выполнения\" -> выбрать аппаратный ускоритель \"GPU\". При этом, следующая ячейка, проверяющая доступность CUDA (платформы, использующей графические ускорители), должна возвращать `True`"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "XtIrqKT4GSXf",
"outputId": "376ab52b-e0d9-4c48-a01a-0c1191d9fa5f"
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.cuda.is_available()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vXAr_sdxFSKm"
},
"source": [
"# Загрузим данные\n",
"\n",
"В соответствии с вариантом"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Krp456VbE7yP",
"outputId": "bd470eb1-d3e8-4059-a5cb-f24831c3d9dd"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2024-04-05 17:50:23-- http://uit.mpei.ru/git/main/TDA/raw/branch/master/assets/poems/pushkin.txt\n",
"Resolving uit.mpei.ru (uit.mpei.ru)... 193.233.68.149\n",
"Connecting to uit.mpei.ru (uit.mpei.ru)|193.233.68.149|:80... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 1048627 (1.0M) [text/plain]\n",
"Saving to: ‘poems.txt’\n",
"\n",
"poems.txt 100%[===================>] 1.00M 625KB/s in 1.6s \n",
"\n",
"2024-04-05 17:50:25 (625 KB/s) - ‘poems.txt’ saved [1048627/1048627]\n",
"\n"
]
}
],
"source": [
"!wget -O poems.txt http://uit.mpei.ru/git/main/TDA/raw/branch/master/assets/poems/pushkin.txt\n",
"\n",
"# Маяковский: http://uit.mpei.ru/git/main/TDA/raw/branch/master/assets/poems/mayakovskiy.txt\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "BFnX81iNFFEf",
"outputId": "f2efb98f-d09e-49d6-f8b4-2952ae48c075"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Количество стихов: 720\n",
"\n",
"Пример стиха:\n",
"\n",
"Возможно ль? вместо роз, Амуром насажденных,\n",
"Тюльпанов гордо наклоненных,\n",
"Душистых ландышей, ясминов и лилей,\n",
"Которых ты всегда любила\n",
"И прежде всякой день носила\n",
"На мраморной груди твоей —\n",
"Возможно ль, милая Климена,\n",
"Какая странная во вкусе перемена!..\n",
"Ты любишь обонять не утренний цветок,\n",
"А вредную траву зелену,\n",
"Искусством превращенну\n",
"В пушистый порошок! —\n",
"Пускай уже седой профессор Геттингена,\n",
"На старой кафедре согнувшися дугой,\n",
"Вперив в латинщину глубокой разум свой,\n",
"Раскашлявшись, табак толченый\n",
"Пихает в длинный нос иссохшею рукой;\n",
"Пускай младой драгун усатый\n",
"Поутру, сидя у окна,\n",
"С остатком утреннего сна,\n",
"Из трубки пенковой дым гонит сероватый;\n",
"Пускай красавица шестидесяти лет,\n",
"У Граций в отпуску, и у любви в отставке,\n",
"Которой держится вся прелесть на подставке,\n",
"Которой без морщин на теле места нет,\n",
"Злословит, молится, зевает\n",
"И с верным табаком печали забывает, —\n",
"А ты, прелестная!.. но если уж табак\n",
"Так нравится тебе – о пыл воображенья! —\n",
"Ах! если, превращенный в прах,\n",
"И в табакерке, в заточеньи,\n",
"Я в персты нежные твои попасться мог,\n",
"Тогда б в сердечном восхищеньи\n",
"Рассыпался на грудь под шелковый платок\n",
"И даже… может быть… Но что! мечта пустая.\n",
"Не будет этого никак.\n",
"Судьба завистливая, злая!\n",
"Ах, отчего я не табак!..\n",
"\n",
"\n"
]
}
],
"source": [
"# Загружаем текст из файла.\n",
"# Стихотворения в файле разделены токеном '</s>'\n",
"\n",
"with open(\"poems.txt\") as file:\n",
" data = file.read().split(\"</s>\\n\\n\")\n",
"print(f\"Количество стихов: {len(data)}\\n\", f\"Пример стиха:\\n\\n{data[10]}\", sep=\"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "urxR6VN2FbVl"
},
"source": [
"# Подготовка данных и сводные статистики"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "OhWos8xuFZZj",
"outputId": "635e93d6-6614-4c2c-c6c0-b0c7e70ea6ae"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Количество уникальных символов: 143\n",
"{0: 'PAD', 1: 'EOS', 2: 'a', 3: 'g', 4: ';', 5: 'R', 6: 'у', 7: 'И', 8: 'д', 9: 'ю', 10: 'V', 11: '?', 12: 'd', 13: 'з', 14: 'ы', 15: '–', 16: '(', 17: 'H', 18: 'Г', 19: ':', 20: 'm', 21: 'é', 22: 'ж', 23: 'c', 24: 'ц', 25: 'l', 26: 'Ф', 27: 'ф', 28: 'â', 29: 'п', 30: 'b', 31: 'г', 32: 'k', 33: 'B', 34: 'S', 35: \"'\", 36: 'z', 37: 'р', 38: 'ъ', 39: 'ь', 40: '!', 41: '\\n', 42: 'й', 43: 'Б', 44: '\"', 45: 'н', 46: '_', 47: 'P', 48: 'к', 49: 'F', 50: '»', 51: '*', 52: '—', 53: 'Ц', 54: 'L', 55: 'ê', 56: 'щ', 57: ')', 58: 's', 59: 'y', 60: 'С', 61: 'Ш', 62: 'Р', 63: 'э', 64: 'i', 65: 'x', 66: 'У', 67: 'è', 68: 'à', 69: 'p', 70: 'л', 71: 'T', 72: 'I', 73: 'û', 74: 'в', 75: '„', 76: 'Z', 77: 'П', 78: 'ё', 79: 'Л', 80: 'ш', 81: 'М', 82: '…', 83: '-', 84: 'З', 85: 'n', 86: '.', 87: 'В', 88: 'х', 89: 'с', 90: 'Ю', 91: 'C', 92: ' ', 93: 'е', 94: 'j', 95: 'Х', 96: 'а', 97: 'Н', 98: 'Д', 99: 'M', 100: 'и', 101: ',', 102: 'б', 103: '<', 104: '>', 105: 'А', 106: 'Т', 107: 'N', 108: 'о', 109: '«', 110: '\\xa0', 111: 'o', 112: 'Й', 113: 'Q', 114: 'U', 115: 'W', 116: 'ç', 117: 'т', 118: 'Е', 119: 'O', 120: 'О', 121: 'ч', 122: 'e', 123: 'u', 124: 'f', 125: 'D', 126: 'E', 127: 'К', 128: 'v', 129: 'Ж', 130: 'Щ', 131: 'м', 132: 'A', 133: 'Ч', 134: 'h', 135: 'Я', 136: 'ô', 137: 'J', 138: 't', 139: 'я', 140: 'r', 141: 'q', 142: 'Э'}\n"
]
}
],
"source": [
"# Составляем словарь уникальных токенов\n",
"vocab = [\"PAD\", \"EOS\",] + list(set(\"\".join(data))) #список уникальных символов.\n",
"\n",
"# Формируем два словаря, реализующие перевод символов в их индексы и обратно\n",
"id2char = dict(enumerate(vocab)) #словарь индексов в символы\n",
"char2id = {char: ind for ind, char in id2char.items()} #словарь символов в индексы\n",
"print(f\"Количество уникальных символов: {len(vocab)}\", id2char, sep=\"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xeTO4fBQFfBS"
},
"source": [
"Рассмотрим длины текстов"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "J-u-IxOeFZXY",
"outputId": "3ac61929-c08f-447f-df37-0f65030d3a57"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Максимальная длина текста: 8948\n"
]
}
],
"source": [
"lengths = list(map(len, data))\n",
"print(\"Максимальная длина текста: \", max(lengths))"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 564
},
"id": "Ym7S8tmNFZUg",
"outputId": "67470c7b-ad81-4445-85d0-d3b1a4621e33"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABHUAAAIjCAYAAACNlSf9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIG0lEQVR4nO3deZhWdf0//tewDeswss2AIqCgqKyi4iguCYJIGmrmQoriT1OhUlzQ3EBNyBZTI1xSSJMwLU3FUESBNFyTFDFywVBhwCX2GAfm/P7wy/1hHLYx4b4PPh7XdV8X9/u873NeZ+5XxDx9n3PykiRJAgAAAIBUqZHtAgAAAACoPqEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AOArNmHChMjLy9vk64MPPsh2iQAA7ABqZbsAANhRXXvttdGuXbsq402aNMlCNQAA7GiEOgCwjfTv3z/222+/bJcBAMAOyuVXAJAl6y/Teu+99zJjFRUV0aVLl8jLy4sJEyZUmv/Pf/4zvvOd70Tz5s2jXr16seeee8YVV1wREREjR47c7CVfeXl5MX369My+HnjggejRo0fUq1cvmjVrFt/97nfjww8/rHS8M844Y6P7ad++fWZO27Zt45vf/GY8+eST0a1bt6hbt27svffe8ac//anSvj799NO4+OKLo3PnztGwYcMoKCiI/v37xz/+8Y9K86ZPn545zuzZsytt+/DDD6NmzZqRl5cXDz74YJU6u3XrVuVnPHr06MjLy4uGDRtWGh8/fnwcccQR0aJFi8jPz4+99947xo0bV+Xzm9O2bduN/ny++L1FbPr7OeOMM6rM2dDKlSujuLi4yvd3+OGHR6dOnaoc52c/+1mVnvqiTX2vG742/Pxf/vKXOOSQQ6JBgwbRqFGjGDBgQLzxxhtV9tm2bdtKY7/73e+iRo0aMWbMmErj2e7jnXbaKQ4//PD461//usmfEQCkhZU6AJBD7r333nj99derjL/22mtxyCGHRO3ateOcc86Jtm3bxjvvvBOPPvpo/PjHP47jjz++Uthy4YUXxl577RXnnHNOZmyvvfaKiM/DpDPPPDP233//GD16dCxevDhuvvnmeO655+LVV1+NwsLCzGfy8/PjN7/5TaVaGjVqVOn9W2+9FSeddFKce+65MXjw4Bg/fnyceOKJMWXKlDjyyCMjIuLdd9+Nhx9+OE488cRo165dLF68OG6//fY47LDDYu7cudGqVatK+6xbt26MHz8+br755szYb3/726hTp06sWbOmys+nVq1a8cYbb8Srr74a3bt3z4xPmDAh6tatW2X+uHHjYp999oljjz02atWqFY8++micf/75UVFREUOHDq0yf1O6desWF110UUREzJ8/P66++urNzr/33nszf77wwgu3uP+f//znsXjx4q2uZ2t873vfiz59+mTen3baaXHcccfF8ccfnxlr3rx5RHxe7+DBg6Nfv37xk5/8JFavXh3jxo2LXr16xauvvlolyFnvySefjCFDhsSwYcPisssuy4xnq4+bNWsWN910U0REfPDBB3HzzTfH0UcfHe+//36leQCQOgkA8JUaP358EhHJSy+9tFXz5s+fnyRJkqxZsybZddddk/79+ycRkYwfPz4z99BDD00aNWqU/Pvf/660j4qKio3uu02bNsngwYOrjH/22WdJixYtkk6dOiX//e9/M+OPPfZYEhHJ1VdfnRkbPHhw0qBBg82eQ5s2bZKISP74xz9mxpYtW5a0bNky6d69e2ZszZo1ybp16yp9dv78+Ul+fn5y7bXXZsaeeeaZJCKSU045JWnatGlSVlaW2dahQ4fk1FNPTSIieeCBB6rUecwxxyTDhg3LjP/1r39N6tWrlwwcOLDKeaxevbrKufTr1y/ZbbfdNnu+G2rVqlXyzW9+M/P+pZdeqvK9rXfFFVckeXl5lca++B1dc801yYb/NFuyZEnSqFGjTD8888wzmW2HHXZYss8++1Q5zk9/+tNKPbU1IiK55pprqoyvWLEiKSwsTM4+++xK46WlpUnjxo0rjQ8ePDhp06ZNkiRJ8vLLLycNGzZMTjzxxCrfebb6eH1t691xxx1JRCQvvvjiRo8LAGnh8isAyBFjx46NTz75JK655ppK4x999FHMnDkzhgwZErvuumulbV+8XGdLXn755ViyZEmcf/75lVawDBgwIDp27BiTJ0+udt2tWrWK4447LvO+oKAgTj/99Hj11VejtLQ0Ij5f8VOjxuf/7Fi3bl188skn0bBhw9hzzz3j73//e5V9HnPMMZGXlxePPPJIRET89a9/jQ8++CBOOumkTdYxZMiQmDhxYpSVlUXE55dYHX/88dG4ceMqc+vVq5f587Jly+Ljjz+Oww47LN59991YtmzZVp33mjVrNroKaGM+++yzyM/P36q561133XXRuHHj+MEPfrDR7evWrYuPP/640mv16tXVOsbmTJ06NZYuXRqnnHJKpWPUrFkzevbsGc8880yVz7z77rsxYMCA6NatW9x7772Z7zwiu31cUVGRqX/27Nlxzz33RMuWLTOrfgAgrVx+BQA5YNmyZXHDDTfE8OHDo6ioqNK2d999NyJio/dQqa5///vfERGx5557VtnWsWPHePbZZ6u9z/bt21f5pXyPPfaIiIj33nsviouLo6KiIm6++eb49a9/HfPnz49169Zl5jZt2rTKPmvXrh3f/e534+67745vf/vbcffdd8cJJ5wQBQUFm6xjwIABUatWrfjzn/8cAwYMiD/84Q/x8MMPV7rkab3nnnsurrnmmpg1a1aVIGTZsmUbDYI2tG7duli6dOkW5623dOnSKvf12Zz58+fH7bffHuPGjdtkcPTPf/4zc5nUtvDWW29FRMQRRxyx0e1f/C5WrVoV/fr1i8WLF0fTpk2r9EQ2+/j999+v9LNq2bJl/PGPf6zWdwIAuUioAwA54Cc/+UnUqFEjLrnkkvjkk0+yXc5X7oYbboirrroqhgwZEtddd100adIkatSoERdccEFUVFRs9DNDhgyJ7t27x7x58+KBBx7IrNrZlPVB0Pjx42P16tXRtGnTOOKII6qEOu+880707t07OnbsGL/4xS+idevWUadOnXj88cfjpptu2mQ9G1qwYEFUVFRs8p4yX1RaWhrFxcVbNTci4oorrogOHTrE4MGDN3lD37Zt28add95ZaeyBBx6IO+64Y6uPsznrfw733nvvRmuvVavyPyM//vjjaNCgQTz66KMxcODAGD16dJVVZ9lSVFQUv/vd7yLi89Du7rvvjqOOOiqeffbZ6Ny5c5arA4AvT6gDAFm2cOHCuPnmm2P06NHRqFGjKqHObrvtFhERc+bM+Z+P1aZNm4iImDdvXpUVGPPmzctsr4633347kiSptDLjX//6V0REJvR48MEH4xvf+EbcddddlT67dOnSaNas2Ub327lz5+jevXvmSUnf+MY3YsaMGZutZciQIdG1a9d4//33Y/DgwRu9rOfRRx+NsrKyeOSRRypdBrSxy4k25eWXX46I2OpH1s+dOzf23XffrZr76quvxqRJk+Lhhx+OmjVrbnJegwYNKt3wOCKqPDHsf7H77rtHRESLFi2qHGdj6tevH1OmTImOHTvGhRdeGDfccEN85zvfyVzilM0+rlu3bqVzOPbYY6NJkybxq1/9Km6//fb/uR4AyBb31AGALBs1alQUFRXFueeeu9HtzZs3j0MPPTTuvvvuWLBgQaVtSZJU61j77bdftGjRIm677bbMvWciPn9s9ZtvvhkDBgyodv0LFy6Mhx56KPN++fLlcc8990S3bt0yKzxq1qxZpdYHHnigyuOnv2jIkCHx2muvZR5LvSX77LNP9OjRI+bOnVvpceEbWh+UbFjPsmXLYvz48Vvc/4a1FxYWxmGHHbbFuS+//HK88847m7yM6Ysuu+yyOPjgg+PYY4/d6nq2hX79+kVBQUHccMMNUV5eXmX7Rx99VOl98+bNo2PHjhERce2118Yuu+wSZ599dubnnEt9/Nlnn8XatWsrfRYA0shKHQDIsieffDLuu+++qFOnzibn3HLLLdGrV6/Yd99945xzzol27drFe++9F5MnT67W6ozatWvHT37ykzjzzDPjsMMOi1NOOSXzKOi2bdtu1WO2v2iPPfaIs846K1566aUoKiqKu+++OxYvXlwpJPnmN78Z1157bZx55plx0EEHxeuvvx733XdfZvXGppx99tlx4oknbvW9ayIinn766SgrK4smTZpsdHvfvn2jTp06ccwxx8T3vve9WLlyZdx5553RokWLWLRo0Wb3vXjx4rjlllvigQceiEMPPTT++Mc/ZrbNnz8/IiJmzZoV++67b3Tp0iWuvfbauPnmm2O33XaL008/favqf/LJJ+O5557byrPddgoKCmLcuHFx2mmnxb777hsnn3xyNG/ePBYsWBCTJ0+Ogw8+OH71q19t9LP16tWLO+64I/r06RPjxo2L888/PyKy18erVq2qdPnVvffeG2vWrKl0g28ASCOhDgBkWbdu3eKUU07Z7JyuXbvG888/H1dddVWMGzcu1qxZE23atInvfOc71T7eGWecEfXr148xY8bEiBEjokGDBnHcccfFT37ykygsLKz2/jp06BC33nprXHLJJTFv3rxo165d3H///dGvX7/MnB/96EexatWqmDhxYtx///2x7777xuTJk+Oyyy7b7L5r1aq1ycuzNqVBgwbRoEGDTW7fc88948EHH4wrr7wyLr744iguLo7zzjsvmjdvHkOGDNnsvt9888244YYbIiJi5syZMXPmzCpz7rjjjmjZsmV06dIl7rzzzhg4cGBcf/31Ub9+/a2q/1vf+lYcdNBBWzV3Wzv11FOjVatWMWbMmPjpT38aZWVlsfPOO8chhxwSZ5555mY/27t37zjzzDPj8ssvj29961ux8847Z62PP/744zjttNMiIqJhw4axxx57xL333hvf+ta3qn1cAMgleUl117sCAPw/bdu2jU6dOsVjjz2W7VK2i+nTp8c3vvGNzV4udMYZZ0Tbtm1j5MiR268wAOBryT11AAAAAFLI5VcAAFupqKgoBg0atNk5Bx10ULUvGQMA+DJcfgUAfGlft8uvAAByiVAHAAAAIIXcUwcAAAAghYQ6AAAAACmUyhslV1RUxMKFC6NRo0aRl5eX7XIAAAAAvhJJksSKFSuiVatWUaPG5tfipDLUWbhwYbRu3TrbZQAAAABsE++//37ssssum52TylCnUaNGEfH5CRYUFGS5mi+nvLw8nnzyyejbt2/Url072+WAniSn6EdyjZ4kl+hHcol+JNfsCD25fPnyaN26dSb72JxUhjrrL7kqKChIdahTv379KCgoSG2jsWPRk+QS/Uiu0ZPkEv1ILtGP5JodqSe35nYzbpQMAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFKpWqDN69OjYf//9o1GjRtGiRYsYOHBgzJs3r9Kcww8/PPLy8iq9zj333EpzFixYEAMGDIj69etHixYt4pJLLom1a9f+72cDAAAA8DVRqzqTZ8yYEUOHDo39998/1q5dGz/60Y+ib9++MXfu3GjQoEFm3tlnnx3XXntt5n39+vUzf163bl0MGDAgiouL429/+1ssWrQoTj/99Khdu3bccMMNX8EpUR1tL5uc7RLivTEDsl0CAAAApE61Qp0pU6ZUej9hwoRo0aJFvPLKK3HooYdmxuvXrx/FxcUb3ceTTz4Zc+fOjaeeeiqKioqiW7ducd1118WIESNi5MiRUadOnS9xGgAAAABfL9UKdb5o2bJlERHRpEmTSuP33Xdf/O53v4vi4uI45phj4qqrrsqs1pk1a1Z07tw5ioqKMvP79esX5513XrzxxhvRvXv3KscpKyuLsrKyzPvly5dHRER5eXmUl5f/L6eQNevrznb9+TWTrB4/Ivs/Az6XKz0JEfqR3KMnySX6kVyiH8k1O0JPVqf2vCRJvtRv9RUVFXHsscfG0qVL49lnn82M33HHHdGmTZto1apVvPbaazFixIg44IAD4k9/+lNERJxzzjnx73//O5544onMZ1avXh0NGjSIxx9/PPr371/lWCNHjoxRo0ZVGZ84cWKlS7sAAAAA0mz16tVx6qmnxrJly6KgoGCzc7/0Sp2hQ4fGnDlzKgU6EZ+HNut17tw5WrZsGb1794533nkndt999y91rMsvvzyGDx+eeb98+fJo3bp19O3bd4snmKvKy8tj6tSpceSRR0bt2rWzVkenkU9sedI2Nmdkv2yXQOROT0KEfiT36ElyiX4kl+hHcs2O0JPrr07aGl8q1Bk2bFg89thjMXPmzNhll102O7dnz54REfH222/H7rvvHsXFxfHiiy9WmrN48eKIiE3ehyc/Pz/y8/OrjNeuXTu1X9J62T6HsnV5WTv2emn/Dnc02e5J2JB+JNfoSXKJfiSX6EdyTZp7sjp1V+uR5kmSxLBhw+Khhx6Kp59+Otq1a7fFz8yePTsiIlq2bBkRESUlJfH666/HkiVLMnOmTp0aBQUFsffee1enHAAAAICvrWqt1Bk6dGhMnDgx/vznP0ejRo2itLQ0IiIaN24c9erVi3feeScmTpwYRx99dDRt2jRee+21uPDCC+PQQw+NLl26RERE3759Y++9947TTjstbrzxxigtLY0rr7wyhg4dutHVOAAAAABUVa2VOuPGjYtly5bF4YcfHi1btsy87r///oiIqFOnTjz11FPRt2/f6NixY1x00UVxwgknxKOPPprZR82aNeOxxx6LmjVrRklJSXz3u9+N008/Pa699tqv9swAAAAAdmDVWqmzpQdltW7dOmbMmLHF/bRp0yYef/zx6hwaAAAAgA1Ua6UOAAAAALlBqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWqFeqMHj069t9//2jUqFG0aNEiBg4cGPPmzas0Z82aNTF06NBo2rRpNGzYME444YRYvHhxpTkLFiyIAQMGRP369aNFixZxySWXxNq1a//3swEAAAD4mqhWqDNjxowYOnRoPP/88zF16tQoLy+Pvn37xqpVqzJzLrzwwnj00UfjgQceiBkzZsTChQvj+OOPz2xft25dDBgwID777LP429/+Fr/97W9jwoQJcfXVV391ZwUAAACwg6tVnclTpkyp9H7ChAnRokWLeOWVV+LQQw+NZcuWxV133RUTJ06MI444IiIixo8fH3vttVc8//zzceCBB8aTTz4Zc+fOjaeeeiqKioqiW7ducd1118WIESNi5MiRUadOna/u7AAAAAB2UNUKdb5o2bJlERHRpEmTiIh45ZVXory8PPr06ZOZ07Fjx9h1111j1qxZceCBB8asWbOic+fOUVRUlJnTr1+/OO+88+KNN96I7t27VzlOWVlZlJWVZd4vX748IiLKy8ujvLz8fzmFrFlfd7brz6+ZZPX4Edn/GfC5XOlJiNCP5B49SS7Rj+QS/Uiu2RF6sjq1f+lQp6KiIi644II4+OCDo1OnThERUVpaGnXq1InCwsJKc4uKiqK0tDQzZ8NAZ/329ds2ZvTo0TFq1Kgq408++WTUr1//y55CTpg6dWpWj3/jAVk9fEREPP7449kugQ1kuydhQ/qRXKMnySX6kVyiH8k1ae7J1atXb/XcLx3qDB06NObMmRPPPvvsl93FVrv88stj+PDhmffLly+P1q1bR9++faOgoGCbH39bKC8vj6lTp8aRRx4ZtWvXzlodnUY+kbVjrzdnZL9sl0DkTk9ChH4k9+hJcol+JJfoR3LNjtCT669O2hpfKtQZNmxYPPbYYzFz5szYZZddMuPFxcXx2WefxdKlSyut1lm8eHEUFxdn5rz44ouV9rf+6Vjr53xRfn5+5OfnVxmvXbt2ar+k9bJ9DmXr8rJ27PXS/h3uaLLdk7Ah/Uiu0ZPkEv1ILtGP5Jo092R16q7W06+SJIlhw4bFQw89FE8//XS0a9eu0vYePXpE7dq1Y9q0aZmxefPmxYIFC6KkpCQiIkpKSuL111+PJUuWZOZMnTo1CgoKYu+9965OOQAAAABfW9VaqTN06NCYOHFi/PnPf45GjRpl7oHTuHHjqFevXjRu3DjOOuusGD58eDRp0iQKCgri+9//fpSUlMSBBx4YERF9+/aNvffeO0477bS48cYbo7S0NK688soYOnToRlfjAAAAAFBVtUKdcePGRUTE4YcfXml8/PjxccYZZ0RExE033RQ1atSIE044IcrKyqJfv37x61//OjO3Zs2a8dhjj8V5550XJSUl0aBBgxg8eHBce+21/9uZAAAAAHyNVCvUSZItP/66bt26MXbs2Bg7duwm57Rp08YTjwAAAAD+B9W6pw4AAAAAuUGoAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKRQtUOdmTNnxjHHHBOtWrWKvLy8ePjhhyttP+OMMyIvL6/S66ijjqo059NPP41BgwZFQUFBFBYWxllnnRUrV678n04EAAAA4Ouk2qHOqlWromvXrjF27NhNzjnqqKNi0aJFmdfvf//7StsHDRoUb7zxRkydOjUee+yxmDlzZpxzzjnVrx4AAADga6pWdT/Qv3//6N+//2bn5OfnR3Fx8Ua3vfnmmzFlypR46aWXYr/99ouIiFtvvTWOPvro+NnPfhatWrWqbkkAAAAAXzvVDnW2xvTp06NFixax0047xRFHHBHXX399NG3aNCIiZs2aFYWFhZlAJyKiT58+UaNGjXjhhRfiuOOOq7K/srKyKCsry7xfvnx5RESUl5dHeXn5tjiFbW593dmuP79mktXjR2T/Z8DncqUnIUI/knv0JLlEP5JL9CO5ZkfoyerUnpckyZf+rT4vLy8eeuihGDhwYGZs0qRJUb9+/WjXrl2888478aMf/SgaNmwYs2bNipo1a8YNN9wQv/3tb2PevHmV9tWiRYsYNWpUnHfeeVWOM3LkyBg1alSV8YkTJ0b9+vW/bPkAAAAAOWX16tVx6qmnxrJly6KgoGCzc7/ylTonn3xy5s+dO3eOLl26xO677x7Tp0+P3r17f6l9Xn755TF8+PDM++XLl0fr1q2jb9++WzzBXFVeXh5Tp06NI488MmrXrp21OjqNfCJrx15vzsh+2S6ByJ2ehAj9SO7Rk+QS/Ugu0Y/kmh2hJ9dfnbQ1tsnlVxvabbfdolmzZvH2229H7969o7i4OJYsWVJpztq1a+PTTz/d5H148vPzIz8/v8p47dq1U/slrZftcyhbl5e1Y6+X9u9wR5PtnoQN6UdyjZ4kl+hHcol+JNekuSerU3e1n35VXR988EF88skn0bJly4iIKCkpiaVLl8Yrr7ySmfP0009HRUVF9OzZc1uXAwAAALBDqPZKnZUrV8bbb7+deT9//vyYPXt2NGnSJJo0aRKjRo2KE044IYqLi+Odd96JSy+9NNq3bx/9+n1+ic1ee+0VRx11VJx99tlx2223RXl5eQwbNixOPvlkT74CAAAA2ErVXqnz8ssvR/fu3aN79+4RETF8+PDo3r17XH311VGzZs147bXX4thjj4099tgjzjrrrOjRo0f89a9/rXT51H333RcdO3aM3r17x9FHHx29evWKO+6446s7KwAAAIAdXLVX6hx++OGxuQdmPfHElm+826RJk5g4cWJ1Dw0AAADA/7PN76kDAAAAwFdPqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASKFa2S4A2l42OdslxHtjBmS7BAAAAKgWK3UAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkULVDnZkzZ8YxxxwTrVq1iry8vHj44YcrbU+SJK6++upo2bJl1KtXL/r06RNvvfVWpTmffvppDBo0KAoKCqKwsDDOOuusWLly5f90IgAAAABfJ9UOdVatWhVdu3aNsWPHbnT7jTfeGLfcckvcdttt8cILL0SDBg2iX79+sWbNmsycQYMGxRtvvBFTp06Nxx57LGbOnBnnnHPOlz8LAAAAgK+ZWtX9QP/+/aN///4b3ZYkSfzyl7+MK6+8Mr71rW9FRMQ999wTRUVF8fDDD8fJJ58cb775ZkyZMiVeeuml2G+//SIi4tZbb42jjz46fvazn0WrVq3+h9MBAAAA+HqodqizOfPnz4/S0tLo06dPZqxx48bRs2fPmDVrVpx88skxa9asKCwszAQ6ERF9+vSJGjVqxAsvvBDHHXdclf2WlZVFWVlZ5v3y5csjIqK8vDzKy8u/ylPYbtbXne3682smWT1+rsj295ALcqUnIUI/knv0JLlEP5JL9CO5ZkfoyerU/pWGOqWlpRERUVRUVGm8qKgos620tDRatGhRuYhataJJkyaZOV80evToGDVqVJXxJ598MurXr/9VlJ41U6dOzerxbzwgq4fPGY8//ni2S8gZ2e5J2JB+JNfoSXKJfiSX6EdyTZp7cvXq1Vs99ysNdbaVyy+/PIYPH555v3z58mjdunX07ds3CgoKsljZl1deXh5Tp06NI488MmrXrp21OjqNfCJrx84lc0b2y3YJWZcrPQkR+pHcoyfJJfqRXKIfyTU7Qk+uvzppa3yloU5xcXFERCxevDhatmyZGV+8eHF069YtM2fJkiWVPrd27dr49NNPM5//ovz8/MjPz68yXrt27dR+Setl+xzK1uVl7di5JO199FXKdk/ChvQjuUZPkkv0I7lEP5Jr0tyT1am72k+/2px27dpFcXFxTJs2LTO2fPnyeOGFF6KkpCQiIkpKSmLp0qXxyiuvZOY8/fTTUVFRET179vwqywEAAADYYVV7pc7KlSvj7bffzryfP39+zJ49O5o0aRK77rprXHDBBXH99ddHhw4dol27dnHVVVdFq1atYuDAgRERsddee8VRRx0VZ599dtx2221RXl4ew4YNi5NPPtmTrwAAAAC2UrVDnZdffjm+8Y1vZN6vv9fN4MGDY8KECXHppZfGqlWr4pxzzomlS5dGr169YsqUKVG3bt3MZ+67774YNmxY9O7dO2rUqBEnnHBC3HLLLV/B6QAAAAB8PVQ71Dn88MMjSTb9GOy8vLy49tpr49prr93knCZNmsTEiROre2gAAAAA/p+v9J46AAAAAGwfQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAK1cp2AZAL2l42OdslxHtjBmS7BAAAAFLESh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEJCHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAK1cp2AV93nUY+EWXr8rJdBgAAAJAyVuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFLoKw91Ro4cGXl5eZVeHTt2zGxfs2ZNDB06NJo2bRoNGzaME044IRYvXvxVlwEAAACwQ9smK3X22WefWLRoUeb17LPPZrZdeOGF8eijj8YDDzwQM2bMiIULF8bxxx+/LcoAAAAA2GHV2iY7rVUriouLq4wvW7Ys7rrrrpg4cWIcccQRERExfvz42GuvveL555+PAw88cKP7Kysri7Kyssz75cuXR0REeXl5lJeXb4Mz2PbW151fI8lyJeSKbPfy+uNnuw6I0I/kHj1JLtGP5BL9SK7ZEXqyOrXnJUnylaYKI0eOjJ/+9KfRuHHjqFu3bpSUlMTo0aNj1113jaeffjp69+4d//nPf6KwsDDzmTZt2sQFF1wQF1544Sb3OWrUqCrjEydOjPr163+V5QMAAABkzerVq+PUU0+NZcuWRUFBwWbnfuUrdXr27BkTJkyIPffcMxYtWhSjRo2KQw45JObMmROlpaVRp06dSoFORERRUVGUlpZucp+XX355DB8+PPN++fLl0bp16+jbt+8WTzBXlZeXx9SpU+Oql2tEWUVetsshB8wZ2S+rx1/fk0ceeWTUrl07q7WAfiTX6ElyiX4kl+hHcs2O0JPrr07aGl95qNO/f//Mn7t06RI9e/aMNm3axB/+8IeoV6/el9pnfn5+5OfnVxmvXbt2ar+k9coq8qJsnVCHyJle3hH+d8WOQz+Sa/QkuUQ/kkv0I7kmzT1Znbq3+SPNCwsLY4899oi33347iouL47PPPoulS5dWmrN48eKN3oMHAAAAgI3b5qHOypUr45133omWLVtGjx49onbt2jFt2rTM9nnz5sWCBQuipKRkW5cCAAAAsMP4yi+/uvjii+OYY46JNm3axMKFC+Oaa66JmjVrximnnBKNGzeOs846K4YPHx5NmjSJgoKC+P73vx8lJSWbfPIVAAAAAFV95aHOBx98EKecckp88skn0bx58+jVq1c8//zz0bx584iIuOmmm6JGjRpxwgknRFlZWfTr1y9+/etff9VlQOq0vWxyVo+fXzOJGw/IagkAAABUw1ce6kyaNGmz2+vWrRtjx46NsWPHftWHBgAAAPja2Ob31AEAAADgqyfUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSqFa2CwByS6eRT0TZurysHf+9MQOydmwAAIA0sVIHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQArVynYBABtqe9nkbJcQ740ZkO0SAAAAtshKHQAAAIAUEuoAAAAApJBQBwAAACCFhDoAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAAAAAUkioAwAAAJBCQh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIoVrZLgCAytpeNjnbJcR7YwZkuwQAAGALrNQBAAAASCErdQC+IBdWygAAAGyJlToAAAAAKSTUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIVqZbsAAHJP28smZ/X4+TWTuPGArJYAAAA5z0odAAAAgBSyUgeAnNVp5BNRti4va8d/b8yArB17vWyvmorIjZ8DAABVCXUAYBNyIVABAIBNcfkVAAAAQApZqQMAsJWyeUmgy+AAgC+yUgcAAAAghazUAQByXrbvb5RfM4kbD8hqCQAAVQh1AABSINvBVq7IhcvQsv1dCBkBWM/lVwAAAAAplNWVOmPHjo2f/vSnUVpaGl27do1bb701DjjAf3YAgFyS7VUJAJuSC38/5cLqMXKDfiQbsrZS5/7774/hw4fHNddcE3//+9+ja9eu0a9fv1iyZEm2SgIAAABIjayt1PnFL34RZ599dpx55pkREXHbbbfF5MmT4+67747LLrssW2UBAJDDcuG/hOeKTiOfiLJ1edku42vv696T6+/xpB9ZL9v/m/i63XcsK6HOZ599Fq+88kpcfvnlmbEaNWpEnz59YtasWVXml5WVRVlZWeb9smXLIiLi008/jfLy8m1f8DZQXl4eq1evjlrlNWJdhb/8yL5aFUmsXl2hJ8kJ+pFcoyfJJfqRXKIfc8snn3yS7RKi1tpV2T3+/+vJTz75JGrXrp3VWr6sFStWREREkiRbnJuVUOfjjz+OdevWRVFRUaXxoqKi+Oc//1ll/ujRo2PUqFFVxtu1a7fNaoSvo1OzXQBsQD+Sa/QkuUQ/kkv0Y+5o9vNsV5AbdpSeXLFiRTRu3Hizc1LxSPPLL788hg8fnnlfUVERn376aTRt2jTy8tKZBi9fvjxat24d77//fhQUFGS7HNCT5BT9SK7Rk+QS/Ugu0Y/kmh2hJ5MkiRUrVkSrVq22ODcroU6zZs2iZs2asXjx4krjixcvjuLi4irz8/PzIz8/v9JYYWHhtixxuykoKEhto7Fj0pPkEv1IrtGT5BL9SC7Rj+SatPfkllborJeVp1/VqVMnevToEdOmTcuMVVRUxLRp06KkpCQbJQEAAACkStYuvxo+fHgMHjw49ttvvzjggAPil7/8ZaxatSrzNCwAAAAANi1roc5JJ50UH330UVx99dVRWloa3bp1iylTplS5efKOKj8/P6655poql5VBtuhJcol+JNfoSXKJfiSX6EdyzdetJ/OSrXlGFgAAAAA5JSv31AEAAADgfyPUAQAAAEghoQ4AAABACgl1AAAAAFJIqJMlY8eOjbZt20bdunWjZ8+e8eKLL2a7JHYAM2fOjGOOOSZatWoVeXl58fDDD1faniRJXH311dGyZcuoV69e9OnTJ956661Kcz799NMYNGhQFBQURGFhYZx11lmxcuXKSnNee+21OOSQQ6Ju3brRunXruPHGG7f1qZFCo0ePjv333z8aNWoULVq0iIEDB8a8efMqzVmzZk0MHTo0mjZtGg0bNowTTjghFi9eXGnOggULYsCAAVG/fv1o0aJFXHLJJbF27dpKc6ZPnx777rtv5OfnR/v27WPChAnb+vRImXHjxkWXLl2ioKAgCgoKoqSkJP7yl79ktutFsmnMmDGRl5cXF1xwQWZMT7I9jRw5MvLy8iq9OnbsmNmuH9nePvzww/jud78bTZs2jXr16kXnzp3j5Zdfzmz3e80GEra7SZMmJXXq1Enuvvvu5I033kjOPvvspLCwMFm8eHG2SyPlHn/88eSKK65I/vSnPyURkTz00EOVto8ZMyZp3Lhx8vDDDyf/+Mc/kmOPPTZp165d8t///jcz56ijjkq6du2aPP/888lf//rXpH379skpp5yS2b5s2bKkqKgoGTRoUDJnzpzk97//fVKvXr3k9ttv316nSUr069cvGT9+fDJnzpxk9uzZydFHH53suuuuycqVKzNzzj333KR169bJtGnTkpdffjk58MADk4MOOiizfe3atUmnTp2SPn36JK+++mry+OOPJ82aNUsuv/zyzJx33303qV+/fjJ8+PBk7ty5ya233prUrFkzmTJlynY9X3LbI488kkyePDn517/+lcybNy/50Y9+lNSuXTuZM2dOkiR6kex58cUXk7Zt2yZdunRJfvjDH2bG9STb0zXXXJPss88+yaJFizKvjz76KLNdP7I9ffrpp0mbNm2SM844I3nhhReSd999N3niiSeSt99+OzPH7zX/R6iTBQcccEAydOjQzPt169YlrVq1SkaPHp3FqtjRfDHUqaioSIqLi5Of/vSnmbGlS5cm+fn5ye9///skSZJk7ty5SUQkL730UmbOX/7ylyQvLy/58MMPkyRJkl//+tfJTjvtlJSVlWXmjBgxItlzzz238RmRdkuWLEkiIpkxY0aSJJ/3X+3atZMHHnggM+fNN99MIiKZNWtWkiSfB5U1atRISktLM3PGjRuXFBQUZHrw0ksvTfbZZ59KxzrppJOSfv36betTIuV22mmn5De/+Y1eJGtWrFiRdOjQIZk6dWpy2GGHZUIdPcn2ds011yRdu3bd6Db9yPY2YsSIpFevXpvc7veaylx+tZ199tln8corr0SfPn0yYzVq1Ig+ffrErFmzslgZO7r58+dHaWlppd5r3Lhx9OzZM9N7s2bNisLCwthvv/0yc/r06RM1atSIF154ITPn0EMPjTp16mTm9OvXL+bNmxf/+c9/ttPZkEbLli2LiIgmTZpERMQrr7wS5eXllXqyY8eOseuuu1bqyc6dO0dRUVFmTr9+/WL58uXxxhtvZOZsuI/1c/ydyqasW7cuJk2aFKtWrYqSkhK9SNYMHTo0BgwYUKVv9CTZ8NZbb0WrVq1it912i0GDBsWCBQsiQj+y/T3yyCOx3377xYknnhgtWrSI7t27x5133pnZ7veayoQ629nHH38c69atq/QXXkREUVFRlJaWZqkqvg7W99fmeq+0tDRatGhRaXutWrWiSZMmleZsbB8bHgO+qKKiIi644II4+OCDo1OnThHxeb/UqVMnCgsLK839Yk9uqd82NWf58uXx3//+d1ucDin1+uuvR8OGDSM/Pz/OPffceOihh2LvvffWi2TFpEmT4u9//3uMHj26yjY9yfbWs2fPmDBhQkyZMiXGjRsX8+fPj0MOOSRWrFihH9nu3n333Rg3blx06NAhnnjiiTjvvPPiBz/4Qfz2t7+NCL/XfFGtbBcAwI5v6NChMWfOnHj22WezXQpfY3vuuWfMnj07li1bFg8++GAMHjw4ZsyYke2y+Bp6//3344c//GFMnTo16tatm+1yIPr375/5c5cuXaJnz57Rpk2b+MMf/hD16tXLYmV8HVVUVMR+++0XN9xwQ0REdO/ePebMmRO33XZbDB48OMvV5R4rdbazZs2aRc2aNavcLX7x4sVRXFycpar4OljfX5vrveLi4liyZEml7WvXro1PP/200pyN7WPDY8CGhg0bFo899lg888wzscsuu2TGi4uL47PPPoulS5dWmv/FntxSv21qTkFBgX+IUkmdOnWiffv20aNHjxg9enR07do1br75Zr3IdvfKK6/EkiVLYt99941atWpFrVq1YsaMGXHLLbdErVq1oqioSE+SVYWFhbHHHnvE22+/7e9ItruWLVvG3nvvXWlsr732ylwS6PeayoQ621mdOnWiR48eMW3atMxYRUVFTJs2LUpKSrJYGTu6du3aRXFxcaXeW758ebzwwguZ3ispKYmlS5fGK6+8kpnz9NNPR0VFRfTs2TMzZ+bMmVFeXp6ZM3Xq1Nhzzz1jp5122k5nQxokSRLDhg2Lhx56KJ5++ulo165dpe09evSI2rVrV+rJefPmxYIFCyr15Ouvv17p/5SnTp0aBQUFmf+zLykpqbSP9XP8ncqWVFRURFlZmV5ku+vdu3e8/vrrMXv27Mxrv/32i0GDBmX+rCfJppUrV8Y777wTLVu29Hck293BBx8c8+bNqzT2r3/9K9q0aRMRfq+pItt3av46mjRpUpKfn59MmDAhmTt3bnLOOeckhYWFle4WD1/GihUrkldffTV59dVXk4hIfvGLXySvvvpq8u9//ztJks8f/VdYWJj8+c9/Tl577bXkW9/61kYf/de9e/fkhRdeSJ599tmkQ4cOlR79t3Tp0qSoqCg57bTTkjlz5iSTJk1K6tevn7pH/7HtnXfeeUnjxo2T6dOnV3pE6urVqzNzzj333GTXXXdNnn766eTll19OSkpKkpKSksz29Y9I7du3bzJ79uxkypQpSfPmzTf6iNRLLrkkefPNN5OxY8d6RCpVXHbZZcmMGTOS+fPnJ6+99lpy2WWXJXl5ecmTTz6ZJIleJPs2fPpVkuhJtq+LLroomT59ejJ//vzkueeeS/r06ZM0a9YsWbJkSZIk+pHt68UXX0xq1aqV/PjHP07eeuut5L777kvq16+f/O53v8vM8XvN/xHqZMmtt96a7LrrrkmdOnWSAw44IHn++eezXRI7gGeeeSaJiCqvwYMHJ0ny+eP/rrrqqqSoqCjJz89PevfuncybN6/SPj755JPklFNOSRo2bJgUFBQkZ555ZrJixYpKc/7xj38kvXr1SvLz85Odd945GTNmzPY6RVJkY70YEcn48eMzc/773/8m559/frLTTjsl9evXT4477rhk0aJFlfbz3nvvJf3790/q1auXNGvWLLnooouS8vLySnOeeeaZpFu3bkmdOnWS3XbbrdIxIEmSZMiQIUmbNm2SOnXqJM2bN0969+6dCXSSRC+SfV8MdfQk29NJJ52UtGzZMqlTp06y8847JyeddFLy9ttvZ7brR7a3Rx99NOnUqVOSn5+fdOzYMbnjjjsqbfd7zf/JS5Ikyc4aIQAAAAC+LPfUAQAAAEghoQ4AAABACgl1AAAAAFJIqAMAAACQQkIdAAAAgBQS6gAAAACkkFAHAAAAIIWEOgAAAAApJNQBAAAASCGhDgAAAEAKCXUAIKWWLl0aeXl5VV6FhYXZLg0AgO1AqAMAKffHP/4xFi1aFIsWLYpf/vKX2S4HAIDtRKgDACm1du3aiIho2rRpFBcXR3FxcTRu3Hijc88444wqK3ouuOCCzPa8vLx4+OGHM+/vuuuuKnPatm1bJTQ644wzYuDAgZn3U6ZMiV69ekVhYWE0bdo0vvnNb8Y777yzyXPYWF3rX2eccUZERFRUVMTo0aOjXbt2Ua9evejatWs8+OCDmX1Mnz498vLyYunSpRER8Z///Ce6dOkSp59+eiRJktnHjTfeGO3bt4/8/PzYdddd48c//nHm3Df1mj59ekREjBgxIvbYY4+oX79+7LbbbnHVVVdFeXn5Js/rvffeq7SfJk2axPHHHx+ffPLJJj8TEXH33XfHPvvsE/n5+dGyZcsYNmxYZtvWfkd5eXnx97//PTNWXl4eRUVFkZeXF++9915EREyYMCFTW82aNaNVq1YxYsSIqKioyHxuxowZccABB2RqueyyyzI9t96G+1n/6tatW2b7lvrhnnvuiYYNG8Zbb72VGTv//POjY8eOsXr16s3+rAAAoQ4ApFZZWVlEROTn529xbpIkcdRRR2VW9JSUlGxy7qpVq+Kqq66Khg0bVrumVatWxfDhw+Pll1+OadOmRY0aNeK4446rFBZs6Oabb87U9J3vfCe+853vZN7ffPPNERExevTouOeee+K2226LN954Iy688ML47ne/GzNmzKiyv5UrV8bRRx8du+22W9x9992Rl5cXERGXX355jBkzJq666qqYO3duTJw4MYqKiiIiMsdbtGhRRFRe+XTQQQdFRESjRo1iwoQJMXfu3Lj55pvjzjvvjJtuummLP4+nnnoqFi1aFJMnT44XX3wxbrzxxk3OHTduXAwdOjTOOeeceP311+ORRx6J9u3bb/LnvKnvaOedd4477rgj8/6hhx6K2rVrV5lXUFAQixYtigULFsRNN90UN954YzzxxBMREfHhhx/G0UcfHfvvv3/84x//iHHjxsVdd90V119/faV9JEmS2c+iRYvioosuqlLn5vrh9NNPj6OPPjoGDRoUa9eujcmTJ8dvfvObuO+++6J+/fqb/FkBAJ+rle0CAIAv59NPP42IzwOHLSkvL4+GDRtGcXFxRETUqVNnk3NvvPHG2HvvvausytgaJ5xwQqX3d999dzRv3jzmzp0bnTp1qjK/cePGmdVF9erVi4jI1BjxeXB1ww03xFNPPZUJonbbbbd49tln4/bbb4/DDjus0txvf/vbUb9+/bj//vujVq3P/5mzYsWKuPnmm+NXv/pVDB48OCIidt999+jVq1eV40VENGnSpMrYlVdemflz27Zt4+KLL45JkybFpZdeutmfx/pVVOvPb1MrqSIirr/++rjooovihz/8YWZs//333+jczX1Hp512Wtx5553x85//PBo0aBB33HFHDBkyJK677rpK8/Ly8jK1tWvXLmrUqJGp79e//nW0bt06fvWrX0VeXl507NgxFi5cGCNGjIirr746atT4/L8LlpeXR506dTL7+WLItDX9cPvtt0eXLl3iBz/4QfzpT3+KkSNHRo8ePTb5cwIA/o+VOgCQUh9++GFERLRs2XKLc5cvXx4NGjTY4ryFCxfGL37xi/j5z3++0e0jRoyIhg0bZl733Xdfpe1vvfVWnHLKKbHbbrtFQUFBtG3bNiIiFixYsMVjb8zbb78dq1evjiOPPLLSce+5554ql3UNGjQopk2bFocddlil1UtvvvlmlJWVRe/evb9UDRER999/fxx88MFRXFwcDRs2jCuvvHKrzumggw6Khg0bRsuWLaN169ZVVrKst2TJkli4cOFW1bil76ioqCgOP/zwmDRpUrzzzjsxd+7cOOaYY6rMW7ZsWTRs2DDq1asXBx54YIwYMSKzMunNN9+MkpKSzEqniIiDDz44Vq5cGR988EFmbEt9tTX9sNNOO8Vdd90V48aNi9133z0uu+yyLf4MAIDPWakDACk1d+7caN68eTRp0mSLcxcuXBhdunTZ4rwrrrgiTjzxxOjatetGt19yySWZe91EfB7yrFu3LvP+mGOOiTZt2sSdd94ZrVq1ioqKiujUqVN89tlnWz6hjVi5cmVEREyePDl23nnnStu+eNlZaWlp/PGPf4xTTz01jjvuuOjcuXNE/N8KoC9r1qxZMWjQoBg1alT069cvGjduHJMmTdpkqLKh+++/P/baa68oLS2NH/7wh3HxxRfHrbfeWmVedWrc0ncUEXHOOefE1VdfHf/6179i8ODBG738qlGjRvH3v/89kiSJN954I4YMGRI9evSosrpmcxYuXBitWrXa5Pat7YeZM2dGzZo1Y9GiRbFq1aqtWn0GAFipAwCpNW3atMzKis1ZtWpVvPnmm9G9e/fNzps9e3Y8+OCDVe6bsqFmzZpF+/btM68Nf/n+5JNPYt68eXHllVdG7969Y6+99or//Oc/W39CG7H33ntHfn5+LFiwoNJx27dvH61bt64095FHHonjjz8+zj777DjzzDMzlyZ16NAh6tWrF9OmTftSNfztb3+LNm3axBVXXBH77bdfdOjQIf79739v1Wdbt24d7du3j169esWZZ54ZDz300EbnNWrUKNq2bbvFGrfmO4qIOPLII+Ojjz6K2267Lf6//+//2+icGjVqRPv27aNDhw4xcODAOOKIIzL17bXXXjFr1qzMjaYjIp577rlo1KhR7LLLLpmxl156aZN9tbX98Le//S1+8pOfxKOPPhoNGzasdHNoAGDzrNQBgJT573//GxMnToy//OUvMXbs2CgtLc1sW7ZsWSRJEqWlpdG8efN466234tJLL43CwsLo37//Zvf7s5/9LC666KLNrrzYnJ122imaNm0ad9xxR7Rs2TIWLFjwP19K06hRo7j44ovjwgsvjIqKiujVq1csW7YsnnvuuSgoKMjcIyciMiuWxowZE126dIkxY8bElVdeGXXr1o0RI0bEpZdeGnXq1ImDDz44Pvroo3jjjTfirLPO2mINHTp0iAULFsSkSZNi//33j8mTJ28ynPmiTz75JEpLS2PJkiXx+9//Pjp27LjJuSNHjoxzzz03WrRoEf37948VK1bEc889F9///vczc7b2O8rLy4vbbrst3nvvvdh9991j9uzZVeas75MkSeKf//xnzJgxI3M/n/PPPz9++ctfxve///0YNmxYzJs3L6655poYPnx41KhRIz7++OO46aab4rnnntvkiqWt6YcVK1bEaaedFj/4wQ+if//+scsuu8T+++8fxxxzTHz729/e7DkCAEIdAEid+++/P7P64vzzz4/zzz+/ypyWLVvG/PnzY+TIkbF27dp46qmntvg0q0aNGm3xxr+bU6NGjZg0aVL84Ac/iE6dOsWee+4Zt9xySxx++OFfep8REdddd100b948Ro8eHe+++24UFhbGvvvuGz/60Y82Or9BgwZx9913x1FHHRUDBw6MTp06xVVXXRW1atWKq6++OhYuXBgtW7aMc889d6uOf+yxx8aFF14Yw4YNi7KyshgwYEBcddVVMXLkyC1+tk+fPhERUVhYGL169dropVfrDR48ONasWRM33XRTXHzxxdGsWbMqwUZ1vqMjjzxys9uXL18eLVu2jLy8vCgqKooTTzwxLrnkkoj4/Alajz/+eFxyySXRtWvXaNKkSZx11lmZG0bfd9998cQTT8RDDz0UBxxwwEb3vzX98MMf/jAaNGgQN9xwQ0REdO7cOW644Yb43ve+FyUlJVUuuQMAKstLNlxXCwDkvAkTJsSECRNi+vTpm5yTl5cX8+fPz9yYFgCAHY976gBAytSrV2+LN0cuKiqKmjVrbqeKAADIBit1AAAAAFLISh0AAACAFBLqAAAAAKSQUAcAAAAghYQ6AAAAACkk1AEAAABIIaEOAAAAQAoJdQAAAABSSKgDAAAAkEL/Py72nAqk2oOCAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 1400x600 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.rcParams[\"figure.figsize\"] = (14, 6)\n",
"plt.hist(lengths, bins=30, range=[0, 6000])\n",
"plt.xlabel(\"Длина текста в символах\")\n",
"plt.title(\"Гистограмма длин текстов\")\n",
"plt.grid()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "JuD6onVvFZR8",
"outputId": "49d24414-289e-4775-ac16-2f33f8873ac1"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Средняя длина 808.9125\n",
"Медиана длины 453.5\n",
"Мода длины 221\n"
]
}
],
"source": [
"lengths = np.array(lengths)\n",
"print(\"Средняя длина\", np.mean(lengths))\n",
"print(\"Медиана длины\", np.median(lengths))\n",
"print(\"Мода длины\", np.bincount(lengths).argmax())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wjXxroS0IDXu"
},
"source": [
"### Вопрос 1\n",
"Где можно будет использовать знание о параметрах распределения длин в выборке?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zfkSHd-8FlE1"
},
"source": [
"Выберите длину для генерации"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "y-Q7-EPuFZO5"
},
"outputs": [],
"source": [
"# Устанавливаем, сколько символов будет генерировать модель (максимальная длина генерируемого текста)\n",
"MAXLEN = 512 #<your choice here>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TSfv4n08Fpiv"
},
"source": [
"# Преобразование данных"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vtelHtjyFq1N"
},
"source": [
"Создадим функцию для преобразования текста в вектора одинакового размера для подачи в нейросеть. В этой функции добавляется токен EOS - конец последовательности. Если текст короче заданной длины, то добавляется специальный токен PAD. Если текст больше заданной длины, то он обрезается."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"id": "Hf9iaFRKJUGq"
},
"outputs": [],
"source": [
"line_ix = [char2id[c] for c in data[5][:512]]"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "2emYDlcVJnwi"
},
"outputs": [],
"source": [
"data_ix = np.zeros([len(data[5]), 512], 'int64')\n",
"data_ix[5, :len(line_ix)] = line_ix"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "QA5lQz2eJ8Oq",
"outputId": "e5df533b-ff5e-494a-d6ae-b2e8e4ffd442"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0 0 0 ... 0 0 0]\n"
]
}
],
"source": [
"data_ix= np.transpose(data_ix)\n",
"print(data_ix[5])"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "YYGc7vo2FoTw"
},
"outputs": [],
"source": [
"def to_matrix(data, char2id, max_len=None, dtype='int64', batch_first = True):\n",
"\n",
" max_len = max_len if max_len else max(map(len, data))\n",
" data = [text[:max_len] + \" EOS\" for text in data]\n",
" data_ix = np.zeros([len(data), max_len], dtype)\n",
"\n",
" for i in range(len(data)):\n",
" line_ix = [char2id[c] for c in data[i][:max_len]]\n",
" data_ix[i, :len(line_ix)] = line_ix\n",
"\n",
" if not batch_first: # convert [batch, time] into [time, batch]\n",
" data_ix = np.transpose(data_ix)\n",
"\n",
" return data_ix"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "DbkCcqotFoRP",
"outputId": "97e86d8c-26f4-407a-c587-58d0df5867cd"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Исходный текст:\n",
" Так и мне узнать случилось,\n",
"Что за птица Купидон;\n",
"Сердце страстное пленилось;\n",
"Признаюсь – и я влюблен!\n",
"Пролетело счастья время,\n",
"Как, любви не зная бремя,\n",
"Я живал да попевал,\n",
"Как в театре и на балах,\n",
"На гуляньях иль в воксалах\n",
"Легким зефиром летал;\n",
"Как, смеясь во зло Амуру,\n",
"Я писал карикатуру\n",
"На любезный женской пол;\n",
"Но напрасно я смеялся,\n",
"Наконец и сам попался,\n",
"Сам, увы! с ума сошел.\n",
"Смехи, вольность – всё под лавку\n",
"Из Катонов я в отставку,\n",
"И теперь я – Селадон!\n",
"Миловидной жрицы Тальи\n",
"Видел прелести Натальи,\n",
"И уж в сердце – Купидон!\n",
"\n",
"Так, Наталья! признаюся,\n",
"Я тобою полонен,\n",
"В первый раз еще, стыжуся,\n",
"В женски прелести влюблен.\n",
"Целый день, как ни верчуся\n",
"Лишь тобою занят я;\n",
"Ночь придет – и лишь тебя\n",
"Вижу я в пустом мечтаньи,\n",
"Вижу, в легком одеяньи\n",
"Будто милая со мной;\n",
"Робко, сладостно дыханье,\n",
"Белой груди колебанье,\n",
"Снег затмивший белизной,\n",
"И полуотверсты очи,\n",
"Скромный мрак безмолвной ночи —\n",
"Дух в восторг приводят мой!..\n",
"Я один в беседке с нею,\n",
"Вижу… девственну лилею,\n",
"Трепещу, томлюсь, немею…\n",
"И проснулся… вижу мрак\n",
"Вкруг постели одинокой!\n",
"Испускаю вздох глубокой,\n",
"Сон ленивый, томноокой\n",
"Отлетает на крылах.\n",
"Страсть сильнее становится\n",
"И, любовью утомясь,\n",
"Я слабею всякой час.\n",
"Всё к чему-то ум стремится,\n",
"А к чему? – никто из нас\n",
"Дамам в слух того не скажет,\n",
"А уж так и сяк размажет.\n",
"Я – по-свойски объяснюсь.\n",
"\n",
"Все любовники желают\n",
"И того, чего не знают;\n",
"Это свойство их – дивлюсь!\n",
"Завернувшись балахоном,\n",
"С хватской шапкой на бекрень\n",
"Я желал бы Филимоном\n",
"Под вечер, как всюду тень,\n",
"Взяв Анюты нежну руку,\n",
"Изъяснять любовну муку,\n",
"Говорить: она моя!\n",
"Я желал бы, чтоб Назорой\n",
"Ты старалася меня\n",
"Удержать умильным взором.\n",
"Иль седым Опекуном\n",
"Легкой, миленькой Розины,\n",
"Старым пасынком судьбины,\n",
"В епанче и с париком,\n",
"Дерзкой пламенной рукою\n",
"Белоснежну, полну грудь…\n",
"Я желал бы… да ногою\n",
"Моря не перешагнуть.\n",
"И, хоть по уши влюбленный,\n",
"Но с тобою разлученный,\n",
"Всей надежды я лишен.\n",
"\n",
"Но, Наталья! ты не знаешь\n",
"Кто твой нежный Селадон,\n",
"Ты еще не понимаешь,\n",
"Отчего не смеет он\n",
"И надеяться? – Наталья!\n",
"Выслушай еще меня:\n",
"\n",
"Не владетель я Сераля,\n",
"Не арап, не турок я.\n",
"За учтивого китайца,\n",
"Грубого американца\n",
"Почитать меня нельзя,\n",
"Не представь и немчурою,\n",
"С колпаком на волосах,\n",
"С кружкой, пивом налитою,\n",
"И с цыгаркою в зубах.\n",
"Не представь кавалергарда\n",
"В каске, с длинным палашом.\n",
"Не люблю я бранный гром:\n",
"Шпага, сабля, алебарда\n",
"Не тягчат моей руки\n",
"За Адамовы грехи.\n",
"\n",
"– Кто же ты, болтун влюбленный?\n",
"Взглянь на стены возвышенны,\n",
"Где безмолвья вечный мрак;\n",
"Взглянь на окны загражденны,\n",
"На лампады там зажженны…\n",
"Знай, Наталья! – я… монах!\n",
"\n",
"\n",
"Преобразованный текст:\n",
" [106 96 48 92 100 92 131 45 93 92 6 13 45 96 117 39 92 89\n",
" 70 6 121 100 70 108 89 39 101 41 133 117 108 92 13 96 92 29\n",
" 117 100 24 96 92 127 6 29 100 8 108 45 4 41 60 93 37 8\n",
" 24 93 92 89 117 37 96 89 117 45 108 93 92 29 70 93 45 100\n",
" 70 108 89 39 4 41 77 37 100 13 45 96 9 89 39 92 15 92\n",
" 100 92 139 92 74 70 9 102 70 93 45 40 41 77 37 108 70 93\n",
" 117 93 70 108 92 89 121 96 89 117 39 139 92 74 37 93 131 139\n",
" 101 41 127 96 48 101 92 70 9 102 74 100 92 45 93 92 13 45\n",
" 96 139 92 102 37 93 131 139 101 41 135 92 22 100 74 96 70 92\n",
" 8 96 92 29 108 29 93 74 96 70 101 41 127 96 48 92 74 92\n",
" 117 93 96 117 37 93 92 100 92 45 96 92 102 96 70 96 88 101\n",
" 41 97 96 92 31 6 70 139 45 39 139 88 92 100 70 39 92 74\n",
" 92 74 108 48 89 96 70 96 88 41 79 93 31 48 100 131 92 13\n",
" 93 27 100 37 108 131 92 70 93 117 96 70 4 41 127 96 48 101\n",
" 92 89 131 93 139 89 39 92 74 108 92 13 70 108 92 105 131 6\n",
" 37 6 101 41 135 92 29 100 89 96 70 92 48 96 37 100 48 96\n",
" 117 6 37 6 41 97 96 92 70 9 102 93 13 45 14 42 92 22\n",
" 93 45 89 48 108 42 92 29 108 70 4 41 97 108 92 45 96 29\n",
" 37 96 89 45 108 92 139 92 89 131 93 139 70 89 139 101 41 97\n",
" 96 48 108 45 93 24 92 100 92 89 96 131 92 29 108 29 96 70\n",
" 89 139 101 41 60 96 131 101 92 6 74 14 40 92 89 92 6 131\n",
" 96 92 89 108 80 93 70 86 41 60 131 93 88 100 101 92 74 108\n",
" 70 39 45 108 89 117 39 92 15 92 74 89 78 92 29 108 8 92\n",
" 70 96 74 48 6 41 7 13 92 127 96 117 108 45 108 74 92 139\n",
" 92 74 92 108 117 89 117 96 74 48 6 101 41 7 92 117 93 29\n",
" 93 37 39 92 139 92 15 92 60 93 70 96 8 108 45 40 41 81\n",
" 100 70 108 74 100 8 45 108 42 92 22 37 100 24 14 92 106 96\n",
" 70 39 100 41 87 100 8 93 70 92 29 37 93 70 93 89 117 100\n",
" 92 97 96 117 96 70 39 100]\n"
]
}
],
"source": [
"# Проверяем работу функции - кодируем один из текстов и смотрим как он выглядит в кодированном виде\n",
"encode = to_matrix(data[:1], char2id, MAXLEN)\n",
"print(\"Исходный текст:\\n\", data[0])\n",
"print(\"Преобразованный текст:\\n\", encode[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nSrGA_gLLh3A"
},
"source": [
"### Вопрос 2\n",
"Пояснить, что хранится в переменной `encode`.\n",
"\n",
"Как будет выглядеть ваша фамилия в кодированном виде?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eQ-s7rzHFu9T"
},
"source": [
"# Подготовка нейросети"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c3qNOFx8Fwec"
},
"source": [
"![](https://raw.githubusercontent.com/tensorflow/text/master/docs/tutorials/images/text_generation_training.png)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "atZoEO6VFyQf"
},
"source": [
"Исходя из архитектуры нейросети на картинке, нужно задать 3 слоя:\n",
" - Embedding\n",
" - GRU\n",
" - Dense\n",
"\n",
"\n",
"Для этого воспользуемся pytorch, ссылки на подробную документацию каждого слоя:\n",
" - [nn.Embedding](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html)\n",
" - [nn.GRU](https://pytorch.org/docs/stable/generated/torch.nn.GRU.html)\n",
" - [nn.Linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) "
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"id": "3rAXL2-QFvM9"
},
"outputs": [],
"source": [
"num_embeddings = len(vocab) #количество эмбеддингов должно соответствовать длине словаря\n",
"embedding_dim = 32 #определяется размерность эмбеддинга\n",
"emb = nn.Embedding(num_embeddings, embedding_dim) # Определяем объект emb как слой эмбеддингов заданного размера"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "mGb6aUqfOmav",
"outputId": "ec17500b-0522-4c56-a6f9-0a5da11f7346"
},
"outputs": [
{
"data": {
"text/plain": [
"143"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\n",
"num_embeddings"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "esvotgUlMADX"
},
"source": [
"### Вопрос 3\n",
"Почему количество эмбеддингов должно соответствовать длине словаря?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hHK3n4noF2OP"
},
"source": [
"В качестве примера пропустим через этот слой первые 5 букв первого текста."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "GvwHFBOWFz-U",
"outputId": "0a58cf39-3151-48d5-846c-6f55d1dfc863"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Размер тензора: torch.Size([5, 32])\n",
"\n",
"tensor([[ 1.5029e+00, -1.7916e+00, 2.2026e+00, -2.2513e+00, -6.9742e-01,\n",
" 3.0790e-01, 6.8104e-01, 7.7460e-01, -6.1624e-01, 1.5153e+00,\n",
" -4.1424e-01, -3.7886e-01, 3.0690e-01, 2.3166e-04, -1.3926e-01,\n",
" -4.1106e-01, -3.6484e-01, 3.3451e-01, -3.3457e+00, -6.2611e-01,\n",
" 8.0940e-01, 8.7477e-01, -9.3242e-02, 1.0553e+00, 1.0860e+00,\n",
" -4.8333e-01, 1.1123e+00, 6.2818e-01, 2.5269e-01, 1.3213e+00,\n",
" 5.3855e-01, 4.6009e-01],\n",
" [ 5.7508e-01, 1.6589e+00, -1.1679e+00, -2.4309e+00, -3.9557e-01,\n",
" 3.8376e-01, -9.8238e-01, -1.1509e+00, 2.2251e+00, -4.6321e-01,\n",
" 6.5630e-01, -1.9887e+00, -6.1972e-01, -1.6304e-01, -9.7050e-01,\n",
" -2.5951e-01, 4.9236e-01, -1.9583e+00, 1.9546e+00, -1.0485e+00,\n",
" 1.2174e-01, 8.9455e-01, -2.3779e+00, 2.3129e+00, -2.7838e-01,\n",
" 9.1432e-01, -2.7290e+00, -1.1626e+00, -8.9565e-01, -8.5091e-03,\n",
" 8.8071e-01, -9.9323e-03],\n",
" [-8.4068e-01, -6.1268e-01, 6.7328e-01, -9.6503e-01, 6.8494e-01,\n",
" -8.4349e-01, 3.5604e-01, 1.7478e+00, -1.7844e-01, -7.7445e-01,\n",
" 1.3449e+00, 1.1115e+00, 1.1597e+00, 7.3386e-01, 9.6546e-01,\n",
" -1.7147e+00, -3.4931e-01, 1.7341e-01, 3.5583e-01, -2.5318e-01,\n",
" -4.7672e-01, -1.4090e-02, -5.9210e-01, 5.4945e-01, 1.6417e-01,\n",
" -4.4186e-01, 7.3914e-01, 1.8834e+00, 3.0256e-01, 6.1556e-01,\n",
" -9.7063e-01, 4.4972e-01],\n",
" [ 5.6335e-01, 1.9862e-01, 3.7290e-01, 9.5874e-01, 6.3197e-01,\n",
" 2.9606e-01, 1.6983e+00, -6.7355e-01, -3.7383e-01, -1.0147e+00,\n",
" -1.2620e-02, 6.6465e-01, 9.1266e-01, 1.2363e+00, 1.9966e+00,\n",
" -1.1470e+00, -5.4097e-01, 1.3002e+00, -1.4012e+00, -8.1303e-01,\n",
" -7.5828e-01, 2.8108e-01, 1.0428e+00, 4.5049e-01, -1.2042e-01,\n",
" -1.2361e+00, 2.9283e-01, -1.6573e+00, 3.8987e-01, -2.1059e-01,\n",
" 1.4592e-01, -2.3350e-01],\n",
" [ 7.5370e-01, -1.6429e+00, -8.4913e-01, 3.6310e-02, 1.4648e+00,\n",
" -9.0935e-01, -2.1241e-01, -3.0972e-01, 6.5774e-01, 6.8460e-01,\n",
" 4.4648e-01, 6.2579e-01, 7.1138e-01, 1.8680e+00, -1.3650e+00,\n",
" 1.2340e-02, -1.2517e-01, 3.1195e-01, 3.0871e-01, 5.0507e-01,\n",
" -1.2373e+00, -1.0363e+00, -4.4852e-01, -8.9384e-01, -4.5717e-01,\n",
" 1.0538e-01, 4.1036e-01, 2.9193e+00, -1.1410e+00, 8.8568e-01,\n",
" -3.3892e-01, -7.0943e-01]], grad_fn=<EmbeddingBackward0>)\n"
]
}
],
"source": [
"# emb_out - 5 первых символов в виде эмбедингов\n",
"emb_out = emb(torch.tensor(encode[0][:5]))\n",
"print(f\"Размер тензора: {emb_out.shape}\\n\")\n",
"print(emb_out)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-YOv6t-qMOQi"
},
"source": [
"### Вопрос 4\n",
"Пояснить, почему получен такой размер `emb_out`?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nkS5ZQ8fF4XP"
},
"source": [
"# Создадим ячейку GRU"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZBfkiOYmM9yr"
},
"source": [
"### Вопрос 5\n",
"Обратиться к документации к [nn.GRU](https://pytorch.org/docs/stable/generated/torch.nn.GRU.html) и ответить на вопрос, за что отчечают параметры `input_size`, `hidden_size`, `num_layers`.\n",
"\n",
"В следующей ячейке задать значения для этих параметров"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5bJ7fRAYFz71",
"outputId": "08fe5314-c7fb-4901-9947-4eabbd4c2326"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Размер output: torch.Size([5, 64])\n",
"Размер h_n: torch.Size([1, 64])\n"
]
}
],
"source": [
"# Определяем ячейку GRU в переменной gru, предварительно задав рамерность скрытого состояния и количество рекуррентных слоев\n",
"input_size = emb.embedding_dim # вход в GRU должен соответствовать размеру эмбеддинга\n",
"hidden_size = 64 #<your choice here>\n",
"num_layers = 1 #<your choice here>\n",
"gru = nn.GRU(input_size, hidden_size, num_layers)\n",
"output, h_n = gru(emb_out)\n",
"print(\"Размер output:\", output.shape)\n",
"print(\"Размер h_n:\", h_n.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4W5Q3O45F6wh"
},
"source": [
"Выходом GRU является 2 тензора:\n",
" - output (используется для классификации)\n",
" - тензор скрытого состояния h_n (используается для последующей передачи во времени)\n",
"\n",
"Теперь используем output для предсказания следующей буквы, пропустив через линейный слой"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "p16NvYn2Fz4-",
"outputId": "a31d5f60-dfe0-4a7e-c078-ee7700750aa6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Размер выходного слоя из нейросети: torch.Size([5, 143])\n"
]
}
],
"source": [
"in_features = gru.hidden_size\n",
"out_features = len(vocab) #предсказываем букву из всего словаря\n",
"linear = nn.Linear(in_features, out_features) # Определяем линейный слой. Почему заданы такие входные и выходные параметры для него?\n",
"linear_out = linear(output) # output - выход GRU\n",
"print(\"Размер выходного слоя из нейросети: \", linear_out.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6no89JxCNwvX"
},
"source": [
"### Вопрос 6\n",
"Что содержится в векторе linear_out?\n",
"\n",
"Определить индекс символа, который наиболее вероятно выдаст ячейка GRU на первом шаге?"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Z2lThHhjOBI2",
"outputId": "50a18b7f-094d-41e1-a728-c47ef9561474"
},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 0.0909, -0.0186, -0.2154, -0.2624, -0.0282, -0.1714, -0.0423, 0.1505,\n",
" 0.0645, 0.0452, -0.0173, -0.0822, 0.1414, 0.1803, -0.2480, -0.1756,\n",
" -0.0447, -0.0136, 0.1097, -0.2245, 0.0189, -0.0559, 0.0215, -0.0040,\n",
" 0.0454, 0.2437, -0.2015, 0.0818, 0.2883, 0.2263, -0.0109, 0.1681,\n",
" -0.0642, -0.0497, 0.1583, -0.0795, -0.0376, 0.0674, -0.1307, 0.0640,\n",
" 0.0966, 0.0140, -0.1366, -0.1685, 0.0397, -0.1600, -0.0497, -0.2244,\n",
" -0.1075, -0.1048, 0.0275, -0.0144, -0.1456, 0.0394, -0.1426, -0.0605,\n",
" -0.0743, 0.1303, 0.0337, 0.1010, -0.0651, -0.1233, 0.0282, 0.2729,\n",
" 0.0485, -0.0911, 0.1994, 0.0610, 0.0911, 0.0940, -0.0973, 0.0747,\n",
" 0.0031, 0.1500, -0.0295, 0.0127, -0.2865, -0.0251, -0.1395, 0.1547,\n",
" 0.0830, 0.0782, -0.0181, 0.1317, -0.0537, 0.1293, -0.2458, 0.1477,\n",
" -0.0610, -0.0592, -0.2878, -0.1622, 0.1336, 0.0987, -0.0834, -0.0815,\n",
" -0.1764, 0.0202, -0.0097, 0.0176, -0.1874, 0.0716, -0.1070, -0.1364,\n",
" 0.1748, 0.1479, -0.0478, 0.0862, 0.0648, 0.1126, -0.0995, -0.1324,\n",
" 0.1023, -0.0247, 0.0439, 0.0212, -0.0608, 0.0402, 0.1468, 0.2311,\n",
" -0.1142, 0.2198, -0.1181, -0.2196, -0.0653, 0.0870, -0.3388, 0.1208,\n",
" 0.0851, 0.0590, 0.1963, 0.0710, 0.1099, 0.1648, 0.0260, -0.1451,\n",
" 0.0012, -0.0973, -0.1058, -0.0628, 0.0585, 0.1859, -0.0573],\n",
" grad_fn=<SelectBackward0>)"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"linear_out[0]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NZnJy28VF894"
},
"source": [
"Теперь определим класс со всеми частями:"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"id": "1GinxRWgF9KG"
},
"outputs": [],
"source": [
"class CharGRULoop(nn.Module): # Описываем инициализатор класса.\n",
" def __init__(self, num_embeddings=52, embedding_dim=16, hidden_size=64, num_layers=1): # В методе __init__ определим архитектуру модели, создав необходимые слои\n",
" super(self.__class__, self).__init__()\n",
" self.emb = nn.Embedding(num_embeddings, embedding_dim)\n",
" self.gru = nn.GRU(embedding_dim, hidden_size, num_layers, batch_first=True)\n",
" self.hid_to_logits = nn.Linear(hidden_size, num_embeddings)\n",
"\n",
" def forward(self, x, hid_state): # Здесь описываем стурктуру сети - как сигнал должен по ней проходить\n",
" x = self.emb(x) # Проходим через эмбеддинг-слой\n",
" if hid_state is not None: # Проходим через GRU, сохраняя hidden state\n",
" h_seq, hid_state = self.gru(x, hid_state)\n",
" else:\n",
" h_seq, hid_state = self.gru(x)\n",
" next_logits = self.hid_to_logits(h_seq) # проходим через полносвязный слой\n",
" next_logp = F.log_softmax(next_logits, dim=-1) # Используем функцию softmax\n",
" return next_logp, hid_state"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1aW5mneXF_cy"
},
"source": [
"Определим модель:"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"id": "_U6Q0oosF_o7"
},
"outputs": [],
"source": [
"model = CharGRULoop(num_embeddings=len(vocab), embedding_dim=64, hidden_size=192, num_layers=2)\n",
"#<your choice here> Можно попробовать создать сеть с другими значениями параметров"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nnS2cn8YGBPw"
},
"source": [
"Количество параметров:"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"id": "zp7I2GVIGCOw"
},
"outputs": [],
"source": [
"!pip -q install torchinfo"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "24-GKj6nGDc4",
"outputId": "8bc88d9d-0105-4ad6-ab6a-5a90c5d2a4bf"
},
"outputs": [
{
"data": {
"text/plain": [
"=================================================================\n",
"Layer (type:depth-idx) Param #\n",
"=================================================================\n",
"CharGRULoop --\n",
"├─Embedding: 1-1 9,152\n",
"├─GRU: 1-2 370,944\n",
"├─Linear: 1-3 27,599\n",
"=================================================================\n",
"Total params: 407,695\n",
"Trainable params: 407,695\n",
"Non-trainable params: 0\n",
"================================================================="
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from torchinfo import summary\n",
"\n",
"summary(model)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gIEsZBJbUNY5"
},
"source": [
"Каждый раз, когда вы вызываете модель (forward), вы передаете некоторый текст и внутреннее состояние. Модель возвращает прогноз для следующего символа и его нового состояния.\n",
"\n",
"![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA2IAAAFKCAYAAAB/z+zgAAAgAElEQVR4Ae29DbQk5VnvW/MBDJ+S3EkykhmCCEdUYkggYcghhjgqCsweEmJQ8WRMSGZPQM+siJkJ07WhFQ0uYHdDEIlizmgmmZ4TVFT0oodz1153VwNX0Tt68B7OXVzvnLVm3cU9jjp6x2RMdqDv/GvX01O7dvXuru767P7ttfbq7qr3q37vv5563s9yHP4gAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEFhO4BMf+vLGndsOTO+Yaj06va21z/+fat33qa0HNy8PzREIlI/AHTd8dcPOm1vX2X9Yuztvbl00PXVw1yltH2x+8uavXFG+q6BEEIgnIH3vmDqwfXrqYLOr422tfTumWvdO39TaUr9ubm18TI5CoJwE4v2Og03pfMfWA+vLWWpKBQEIQCBFAosOauvJ6alWZ4V/T05AitmSFARSJzC9rfVgRMNP62EedCosRM4t6n1ra0/qBSFBCKRI4Kdv3nf+ihoObPeOqdYr4c6HFItAUhBIlcCOrQcum9528KlYm3zKFzmxc+uB++lgSBU9iUEAAmUisOOmA9eeHCU41scYWgPtMD1UZao9yhIlML21NRfR8jPTU60jkWOm5+4nzmuUJL/LQkAdYNNTrZf7aTh0/vj0VOuqspSfckAgSmDn1gO3TE+1pNOuDe7z3fvp6/ati6bDbwhAAAKVJrDzxoOXxxjDEzumDrSmp1r3TU+15MQuNZTbWvsqfdEUfqwJxOjZ9Luo68URs8NRXWs67liD4eIqS2B6atlshcMaJZje2rrdn5I41XohqufpqdaLlb1gCj7WBHZu+8r101OtmNkJB48FHWne9FTrRFTT2OixlgUXB4HJI6Ch/umpg4cixu6wGmdhGtPbWrdGwiwwKhYmxPeyEAg6FqzhFf58QdNvrZzBCMPSB/22g0/ZeT4hUBYC/rTaSGeY1tREy7dz28FayE5rpOFF7HSUEr+LJuBPsZ1qHQ1p1ez0feERL2l8eqqlBpmd1+dCnPaLvibyhwAEIDAUgcUF30uMnAxd7HQWPdTDBlHTCobKlEgQyJBAvKYPHrrjx756TjTb6anWklGxnVsPPhENw28IFE1Am86Eba++RzvLVEa/Y23rwW13TH31kqLLTP4Q6EUgGMENN646Gt2NCx80xiIjZwd3xYXlGAQgAIHKEYj2Nu2cOvBsr4uYnmo9HnYG1PvaKyzHIVAUAX+3z6U9qLGdC+p5jU6N2TF14K6iyk2+EOhFoMcor0Zz90+r4RXTydArLY5DoGgCMet1Xw2PhEXLF+0E1oY10TD8hgAEIFA5AsH0gCW9UtNTraf97WK1ZWzkP7qzkXq1KnfRFHjsCSx7aE8dPBR30dqyPtyx4H8/ufV3XFiOQaBoAtPbWi8t0+upDocTss+apcDOckXXFPmvRCDO7vYaDbN0opsvrdRhbHH4hAAEIFB6Aos7JS6blhhtmPX8TUOs9FU8cQUMRrmWrPvqpdMd21p3Rh1bdU5MHDQuuBIENGV8On5dzRIbra3rNZWxEhdFISeOQMx6804/vS5bx85a3onTDRcMgbEkEGwdu+QhHnVMV/q9c+rgbWMJhouqLIHAWV2q6R6jXFoPFtH3y5W9cAo+EQQWN5g52FxhV1DT/oI62iYCChdZKQKa/h2xu52VOsB6dK6xu22lap3CQgACsQTiGmLR6Ygr/ebFzrFYOVgggSSjXMt6Wada+wssOllDYGACWhMm26wpWtF1jiEn94WBEyQgBHIiELdRx0pZ75hq3RjStN/RIO2vFIdzEIAABCpBQC+vjRq4lRbMVuKiKOREE/A3Lzi1bqajdTVxQOTIRrV/8oXm7MQVB4tjpSag7emDUYZlL8YtdcEp3EQS2LntwHTU9q60y2d0fZg6HugEnkjpcNEQGD8CcUP+esli3JX6D/rFxeD3a473SoYzLj7HIJAHgZgNDWJHueLWRzKVK48aIo8kBKZvam2ZXnz5+NPTU62Xtf6r12Yc01P+lEWbmqjPhSR5ERYCeRCInT4+1bovLu+4Rhs7JsaR4hgEIFBZAtERBD3ooy9LDBbXLn2Px7bWrZW9aAo+lgSSjHJp9CvSK7vAaPBYyqLSFxU31VavZ4g2xvwXPm9btquiV+mLp/BjS0B+RtT+Tm9t3W4XLH0H2l/qd0y1ju+8uXWRheMTAhCAQOUJaGRr+cLvg8f0zjBtKRt9z5hvPNmxqPL1Po4X4I8ehKclLn7v9XLyJ5c6AvFb3I8jJ66pOgSCWQsvL9VqqyNHVg2yYL3N/pMdC7LZ4dGwTq/ZDdW5eko6rgTidk4M9Ht00edYrmedZ4OwcVUE1wWBCScQbNoR7Xla8lAPPeQ9Xh464YIp6eXH7MbVc5Qr2iOrHRRLelkUa8IJ7Nh64LLpqdarIRvcyzZ3j/d7L9OEI+XyS0AgZiptV78xWl/QNMUSFJsiQAACEMiGQLBmZlnPa8ggnlDva3RKTDalIVUIJCcwPdWKjHK1XoxLxZ/GFRk9CE+LiYvDMQgUSUDTsfTC5pA97uW0vsqoQZE1Rd5JCGj3w76dDFtbc9pYLEm6hIUABCBQWQLB9K77tCBW/4vTXw5sX+k9H5W9WAo+VgS0CFwvBrV/jSTEXaC0bGHsE33HkeJY2Qho/a46DfzRhMBG+7Z6qnWfpiLSUVa2GqM8/Qj468GmWjf6SyECTS++4/Hgrk/e/JUr+sXnPAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAApNMoF6fWzvJ18+1jx+BRv35S8bvqriiSSaApie59sfz2tH0eNYrVwUBCCQgoEZYY8a7P0EUgkKg1AR8Tbves6UuJIWDQAICaDoBLIJWggCarkQ1UUgIQCBrArP3zG9puN6JB+pzG7LOi/QhkAeBQNMdelvzoE0eeRBA03lQJo88CaDpPGmTFwQgUFoCs7X2Ew3X6zRd797SFpKCQSABAdP0rOvVEkQjKARKSwBNl7ZqKNiQBND0kOCIBgEIjA+BYGrA0aAh9nf1+ty68bk6rmQSCYQ13XDn/3oSGXDN40Vgqaa9Q+N1dVzNJBIINP0P8j0aLpqeRA1wzRCAgOM43akBi8aw03DbtwMGAlUmsEzTbNpR5eqk7HF2Gk2ji4oTwE5XvAIpPgQgkA4BmxoQ9Ep1Zl3vpXRSJhUIFEMgRtNMTyymKsg1JQJoOiWQJFMaAmi6NFVBQSAAgaIIRKa7aHqA///wPfPXF1Um8oXAKATiNE3nwihEiVs0AWm66bb/3uyz/znj/VXR5SJ/CAxLwNd0zfvHJZpmeuKwOIkHAQhUlcCyqQHd6YneM1W9Jso92QR6apqpXJMtjApfPZqucOVR9FgCaDoWCwchAIFJIxCdGuD3TtW81/X5yN7nLp80Hlxv9QnEatr1NOWW6YnVr96JvAI0PZHVPtYXjabHunq5OAhAYBACcVO4wtMEZCgHSYcwECgLgdgpXN1R3vn/WpZyUg4IDEpgUdPLpnAtTiOfmf8vg6ZDOAiUhYDve9S8fwr7G93vTLktSzVRDghAIGsCPacGdB1XXvCcdR2QfroE+mqa6YnpAie1zAmg6cwRk0HOBNB0zsDJDgIQKCeBXlMDuj1TvOC5nBVHqXoS6Kdppif2RMeJkhJA0yWtGIo1NAE0PTQ6IkIAAuNEoOl6dzZd797gf04NsKbrPRU6du+s254ep2vmWsabQF9N19q3jTcBrm7cCKDpcatRrgdNowEIQAACEQJqfPkjYbX29sgpfkKgkgTQdCWrjUKvQABNrwCHU5UkgKYrWW0UGgIQSJtAmYxhsz530WytvblRm9+mETmVrVlr72u43rMN15truO39mtqgaWaztfatzb3PXaHFv2kzIb1qE6iIpp+RpqVvNF1tveVR+opoGjudhxjGJI+yaFp+x0M179rZ2vwtGrFrzHj3LfU75HsE/sfM/IMKo/CP1efOGZOq4DIgAIEiCRRlDOv1uXVauDs7M/9gY8Z7Obw+LeH3E03X83QdapgVyZK8y0EATZejHihFegTQdHosSakcBIrS9Ofrcxsbbvv2hus92XC94wn9jcVdS09tbPZiY8a7Xw2zclClFBCAQOUI5GkM/REvtz3dcL2nYw1grX244Xov+AayNv+oRr4atfZ2Ndiarndds9a+TSNl6rFSmFnXe2mZEa3NvzLrenua9bnzK1cZFDgVAmg6FYwkUiICaLpElUFRUiGQl6aD10BcpwZTvM/QPqzO3J5+h3yPrv/h7WnU5h9vuN6LDddbWOJ/1NqH8T1SkQaJQGCyCGRtDBffG9LeHhiuaG/SC8pf0xGHpa7pAQ/fM399ozb/aMP1joQM4zGlTYNsWLLVjTcOmm7unb8RTVdXg2mXfBw0jZ1OWxXVTi9rTT9Qn9sQdNoeC/kF8kH0u6VO3ofqc+uHpWizepoz8018j2EpEg8CEHCyMoZqAJ0c9bqr4XqvhozgUa3z0sjWKAZwpWrTw37W9Z4K5ek3yJjPvRK18TqHpserPrkaBzuNCMaOQFZ2ulF//pJg1OpEyA84pEaZphBmta7c72hYnO1jHc6+76EG29hVHhcEAQikRyBtY6j510EPUbgX6gVtwJGVAYyj0ai1r1pcYOstGsVa+zDzuONIjd8xND1+dTrpV4SmJ10B43f9aWtaM2uCTlibMrjgd/zmvHZc0xiX+h7zr4wy62f8ap4rggAElhBIyxhqBCx4WaMZwY6MoozSkgxz/hEYRa07U4NsQZuD5NkgzPlyyc5Jb/QATSOnshDATpelJihHWgTS0vRDdz9/WbCzso1EnVBnsNakp1XWYdIJfI9D5nvoevE9hiFJHAiMOYE0jKG2km+43tHA4JzQ1q+P7H3u8rKgk/ELrtMaiYdkvMtSPsqRLgE0nS5PUiueAJouvg4oQboERtW0nuvaHKPhejYF8ajSzGrZwzBXrzJqk5DQxh4vaOrkMGkRBwIQGFMCoxhD9Tg1XE/vQ7KeqGeK7oVaqZr8d5Sd2ir/GFMVV6JV3XNourp1R8njCaDpeC4crS6BUTTtP8tdz0abOtrYqMwbc8nXOLk5iHaFlq/0Kr5HdXVLySGQOoFhjKHfy7O4EYe9g+OoNuBIvXAZJKhNO4L3h8ggntBLHDPIhiQLJICm0XSB8sskazSNpjMRVoGJDqNpPb+DNeiLs1tmvJer0qhR2UMbiR3H9yhQfGQNgTIRSGoM/S1hF9+h4Y+CaRpimaYCDMo2MOY2krdr0HiEKz8BNO33uqLp8kt14BKiaTQ9sFgqEjCppv21YKdGlRa0C2LVdiT0O7EXX7Uj32Oh6Xp3VqS6KCYEIJAVgSTGsLn3uSu629Hr5YX3zG/Jqlx5pHvyfSK7utMqTxr1PPIkj+wJoOngRaNoOnux5ZQDmkbTOUktt2ySaFq+xsnX4dgMnBfli+RW0AwyCta2+R3B+p5BFiQJAQhUhcCgxlDbz4cM4VyZ52MnYa/pAaGFtIwiJIFX0rDDarqKI7txVYCm46hU+9iwmsZOV7vex7n0A2vabd8eeka3qjYK1qsOtZwjdF34Hr1AcRwC405gEGO4pPem1n5Cw+vjxCXsuFZlrds48U/7WtC046DptFVVbHoJNO2vndGrRLDTxdYZua9MoJ+m/Wl8izsOLi4hGMMRfuz0yhrhLAQmgsBKxlCGUGvAgul7C+M8hK652t3rZAOPSmsfTS9WH5qutIyXFB5No+klghiDH/00HdrYYuHkjoPbx+CSYy8htERiQQ2z2EAchAAExpfASsZQW8IGjZPjmpo4vhQWryw08reg7XHH/XrH9frQ9KmaRdOnWFT5G5o+VXto+hSLKn9bUdNue3/gexytyq6Io9TFrOvVgus9MQnXOwor4kJg7Aj0MobhXppJMgyhxuer2iFy7Cp8Ai4ITS+t5NAOoWh6KZrK/ELTS6sKO72URxV/9dJ097jrHa/6phxJ6iVspz9fn9uYJC5hIQCBChPoGr3Q0H+wMYetNbi1wpeXuOj+dEzX89Q71XQ9b9zWWSQGUsEIaHpppaHppTyq+AtNL601NL2URxV/xWk62MBicWv3vfM3VvG6hi2zNN1wvWeDkbEX8D2GJUk8CFSMQNQYBm+s97eJHec1YStVk3qjTo4IHvMNYm3+0ZXCcq58BND08jpB08uZVOkIml5eW2h6OZMqHYlqWjNvbBfBWbc9XaVrSauswXtaj8r30IY7aaVLOhCAQIkJhI2htu8+9Z6w+cdLXOzMixaMCi7u1hQaLcw8YzIYmQCajkeIpuO5VOEomo6vJTQdz6UKR8OaXtIAmZl/sArlz6qMD98zf30wKtZpuO3bs8qHdCEAgZIQCBvD0Lz7OYbFHSfE43ij/vwlJakyitGHAJruDQhN92ZT5jNounftoOnebMp8Jqxpjf4EjY9ny1zmvMo2OzP/YMDjxCStk8uLL/lAoFQEThlD77PBtICFR/Y+d3mpCllQYfTiyIbrHQoMInO2C6qHpNmi6d7E0HRvNmU+g6Z71w6a7s2mzGdM0829z2nHQK1JX6DDc7HGgvViLwa+xyE6xsusZMoGgREJmDFs1NqLDQ7WRC0h+tDdz1/WcL2JXjO3BEgFfqDplSsJTa/Mp4xn0fTKtYKmV+ZTxrNdTbvt/6IGh0aByljOosrUrM9dZL5HY8a7v6hykC8EIJAxgVPG0NN6qGPN+tz5GWdZueT1MsmgZ2qBaQLlrz403b+O0HR/RmUKgab71waa7s+oTCEimj6K77G8dmZr7VvN9+Ddpsv5cAQCY0EgbAybrnfnWFxUBhfRcL0W0wQyAJtBkmh6MKhoejBOZQiFpgerBTQ9GKcyhELTg9VCd/1crX34sfrcOYPFIhQEIFAZAkuMYX3uosoUPOeCygCe7HE97DfGZrz7cs6e7BIQQNODwULTg3EqQyg0PVgtLGp6/hXs9GC8igyFpgejL03Put5LvqZZOjIYNEJBoEoEQm9z/6cqlbuIsgbvWNMUzhMsKi6iBgbLE00Pxkmh0PTgrIoMiaYHp4+mB2dVZEg0PTh9baDW3dCk1r5q8JiEhEA1CVzmOM5tjuNsn4T/n/3or/6Zelp2/mTzv0/C9Yau8UbHcRIP8zfc9n6/Z8r1nqmQvNH0ZNzPk6JpvUpiYmy0bBZ2Opm1raCdnjhN79n5Zd/3+MVP/+H/HXouj7vftc1xnI3J1LwYOrSlPTs4DwOQOKUnsNZxnF1r1qw57DhOZ5L+L3nbOzvXf//HOm99y6UTdd2q49Wr13zdcZy7HMdR/Q/09/n63EZ/REy7PNXatw4UqZhAgaZPUwN7ouoWTY+tprc4jnNo0vSs651kTa9atfprY2ynJ1bT73/PRzp3/rvPdzZfsXWink+6n08//Yz//aQtU6Ns4D9tZtJwvaPqCJ5129MDRyQgBCpAYIPjOC/aw/0tb3pr5/vfe0PnQzd9rPOTH76T/zFkcNP1t3U2XnBx1/ifcca6/5RkdCw0t/2I5m+XUOMbVq1a9RdoenLu36im165dqxHbgbVZAU3vchxnQZo+79w3YKPH0C5Hn7dRTa9ateaP0fTk2LSoHqr+e9uPfrRz6cWXd04/7Yyu77F69eovJ9G0GmDBjJxjD9Xn1pfQ96BIEEhMQEJ+WQ93NcDu3f14549a/5X/CWGg+j77rHN9o3jaaae1BlWP/wLRUxt3lO39HutXrVr139D0ZN7HYU0HD/mBZF1yTV8vPa9eveZ1Oee/s+8vsNETYqP1PA5r2nGc/QMJ+mSgqmhaDjqanhx7/XtfOtT58Q/uDDfIkmh6rW3cod0UB70XCAeBMhN4Sg94jY78xy/+GQ/3CXq4W4P7C7N/1G2MOY4z8Pb9ofd7lG3jDjQ9gTo2PetzzDStKbavyE7LeQlfJ98nx3lF05NT15NyXz/2wB8M53vcM78lGBXrNNi4o8ztC8o2AAFtYNA5/fR133qi+Sc84CfYeVWPq7SwatXq/zGAbrpBmq7nLRrE9sA9Wt3I2XzxNX3GGeteQ9OT7biYph3HeTWJ1Eqo6Wt1b75p/be/pp7kSXHSuM7l9++4afrN6y94HU0vr+dJ0v6wmm643tNBY2wuiX0nLATKRkCjH513fd+1r0/Sjc+1xhv+0JoxNWYG+lNvlPVMNfc+d8VAkbIN5Gv6yndci8M6wR0Ldo+Piab3yE5/4NqtaBpNh9f2VtlOo2m03LVnF2x427dk4xzHGVzT9ecvsU3DmnvntVsufxCoJIF7Jf7bf+oz3RvCHBg+4xsr48xFa08CY6htsQf+a9ba+0rUM4WmecB37dk4aXrHR+/uXtc42yGubeVnD5pemQ/6qR6fYTXdmPHul++hNWP1+tzAOz8P7NwQEAI5EPCd1k9/6nM84HFe/d0xg4aY3mMy8N8D9bkNDdc7LoP48D3z2lSgyD80jZa79ky7jKHp6jlmONO96wxN92aDbqrJZlhNB9vZv+p3BNfaifyWIp0U8oZAmABOK07ryE6rBDXrerVgVOxQwT1TaBpNo2k00NXAuDnnwzqt2OlqNlLGTb9x1zOKphtu+/bA9ziiXULDDi7fIVAFAjitOCxdh2UUY7hkm+TafKKXNKZ8o6BpNJ26pmdr87ekrNMkyaFpNJ26phvY6S7TuMYBx/JruI7oe6xtuN6hoDGmdy3yB4FKEeABzwO++zAaxRhK9aEXLR4q8C5A02gaTaOBrgbGzaHGTufXQBg37ZT1ekbVtDbrCBpirzIqVqD3RdZDEcBpxWHpOiyjGkN/VMz1gvnahY2KoWk0jabRQFcDZXU+hy0XdpqG2LDaKWu8UTUt75dRsaHaAEQqAQGcVhyWrsOSkjHcZWvFCtI3mkbTaBoNdDVQVudz2HJhp2mIDaudssZLQ9OztfatjIoV5HWR7UgEcFpxWLoOSxrGMNjF6JhvEItZg4Cm0TSaRgNdDZTV+Ry2XNhpGmLDaqes8dLQtDYJa9TmXwkaY6wVG6lpQOQ8CRTmtD70i63OW970Vv//+g98uPvQPPAbz3WPf893vat7fCUD8luPzflxvv+9N3TD69j9M7/V/a28Bk1vpbyGOWfX+qGbPtYtT6904q6lV9i0j6dhDCXepuvdW+CoWGGavvvTD3e1K5ZWP4898Afd42GN2vm4zzjNPNH8k07zl7/qp1ukTlReu9ZB3m0Vdy1x15zFMTQ9mtOq+jU7/e933NfV9C/e/Rvd44PYtV6akZ6la50vUifK365V2u6nxST675dW0vNoejRNS6+m6Xt3P96ta+nbjg9i13pp5pdr/6EjP0bni9SJ8rdr1b3VT2dJ9N8vraTn09J0aJ06a8XybEmQ10gECnNa1UgK3u/TOfusc7tGYs+/n+0el1Ec5Ib+4uef9eO8/Xve44eX47tu3Vn+e7EsvtKqQkMsei1W/jw+0zKGD9Xn1tsb7wvYmaswTet9fKbpt226tKvdn/6Jn+seN432q0+7P37w/R/009HDfc2aNR1751+VGmLRa+l37WmeR9OjOa0hfp0r33FtV9PbfvSjXU2bRvvVm90fSlNh9Vuatg6zKjXEotfS79rTPB+qk5HemzSpdlp6NTv9oz94a1fTm6/a0j1uGu1Xb1YXZpf/3Ud2+WnoOa64VWqIRa+l37Wned7ydhxnJE2H16k3Xe/OkbxjIkMgJwKFOa3mnKkRJqNoPTYyjHo4n3fuG/zeKd3s//GLf+Y/rL8w+0ddo/n5X/m97gM83Hj5vS8d8ns2laYMrsIpDeVneVh6+x+f7yjuz9/5QDetsHHReTUMZUyth0vnlYfSU1w5xIpvaVsccy56lV/hlK4MuMVV2PC1hMuSx/e0jKG025yZbwajYnM5admyKUzT5py94fz1vqZVx6o3ObB2zBpiOmcasrqVDkwLOmcall5/bNsn/d/61H1gGjR9W3oKqxEGlcVGzyx9fUpf0rR71692fmffX3Tvp+g9ofjq0LA40ng4PctP6Vn6lnY4rs6Fr8XC5vWJptNpiEm/6tz6gy+/5Nf3xgsu9jvQTKOqT9W/6tp0b3VvGrX7Q3WiMOYQqydecU2DZuctPWldaSi+nQvrR8ekT41uKKydM40qXWlZ8ZWmzts9Ek7P8guX39JW+uGw4Wux/PL6RNOjadp0J01/+4YLfT1I1/JFzB8RY9Wn6lyalob02+yuacHqwrQlW697QjZWOjINmu4sPRtuPokAACAASURBVKX1QP3Lvm7tXFg/pnd1wNk9p/OmUZVDzwrp0vQa1bjCW35Wfh2ztFXGcN7hawmXJY/vlveoDTE5AQ3XuytYGvFKwe80NZ+ETwisSKAwp9WcM+uF0qiBbniNXF126Tu6UwR0zMLKgJpRkEMrg6ffMib6rmP2Xb/tmMLou42wWXrK+/TTzvDP6bx6eS39nR9z/QahpaNwMnrh/FTOcPybrr+ta8gVz6bsWH5Wfk3rCcdT2A9cu3VJ2uawW3ny+EzTGD5Qn9vQcL2FRYPYvmpFFaZ7sjBNm3Om6YeqU+lFD1E5sHbM6tXCirnVrfQZ1ag0Y/oxLeqY6Tya3r+9+oeX6DY8xUaNOHVyWDpyOqRF5W95KD0Lo091jIS1aulFy6/fFs/S76V/u948PtH0aE6r8TP9Sidy/FTHZrvNrllYacHqVuGiGlU4049pRcdMg9H0LG+FlcbkQCp93VsKa2noc/0b39LtzLA8rnj7Nd0w0rJ19ll66hBTetHy3/5Tn+nGszz0XFBYS1tx7Frz+rRypuG0TqKdNs2YrmRL1ahRHZumrV4trLSp+jW7G9Wo9BCqFz8tHYvqxNKzvE2DapQpfXWOWWPONKdOj2jDLxxGdlzLOyy87hFLz/Kz8stHsXCWdy/956Vn5RNiN9KImFyJEqxTT9ejIbWxJ1CY02oPXd2AMiR6WJuRk8PYyyk146DwMiT6bfF0TD1FchZ1TkbIemP1O+rk6qEtA6eeJD2g5TArPcWRMVNvmQy0jJri6ph6Ui0/lVthNVKg9PUvx1ajZEpPo3pKz67VjLemL1x68eV+3hppk6FV3Oi16Hee/2kaQ98g1tr7glGxJ3O8kwrTtD10rRGv+ra6v/MTdb+O45xSq+NemldvZnREzDQYTU9TIqU/9aRKUzZFUqMF+q3OA2lW56Vf/UuDVk6VQWnr4azw0rH0rTj6ffHbLvM1adcqzaj86sRQ2spbjrrSVY+zzlnapn+73jw+0fRoNsT4mX71Wx0M0oJ0rk+rVwsrbVjd6nxUowonjSieztuIWFQnlp4aUtKoaU5OqNK3Mr3nXdf5tlSalY2WXVcjzcLL1iq+NaykS9lxjQorf8VXepaflV/OsvLW/afnhMIqLYW1tBXHrjWvTytnGg2xSbTTpjvTj9aG2fRx8x2sXi2stKn6Nbsbp3mdswaSjYhFdWLpqeNVjS6bymhTJM3Oq2EljVoZTXdW98pH8a1hJTuve8rKr+Mqr+Wn8iu8tK7OOvlJ9oxQ41NhLW3Tf156DuedmqaDdepN1/Ny9D3ICgJDESjMabWHrm5+G5kyoxNeCK6b1MKa8dMxPdz1YNR3M472wLfwZkwVRmGjDTEzVjqvc5bej39wp/89vDjdDJwMtuVnBiyafjQ9K0+4/DalUQZYTqvyVrqWtl2L0srr3wxxWsawUX/+kmBUbEHfh1Jo8kiFadoeuvpUo0QOoTE1R87q1cKGNdqrIab6t/D61O+oTuy8jSwrTFjz9kDWw9f0pBErhZHTYBq1eyKafjQ9yy9cfjkOumfkRKgBZ/eTpR3Wv5Uh60/jj6aHsyPGT3WoBox0rXpUg0cdWKpjq1cLaxo1zfTSvIVX2gob1Ymdt5kIakwpP0vPnF4bLVAacjIVRh1oplE5u+H0e80+sPzC5dc1qsGpOErXniGWtuIo7Tz/rZxoejjuZgtVt9KxNbj1HDYNWr1aWNOo2cVemrfwCidNRHVi5y096+Cy9KxTVo0m05QaYdKefAare9OopS+fReGt/JZeND+F0b0hf0aNsvD9FE3b8s/j0/JOS9PBSO8JdQTP1tqbk7sRxIBAfgQKc1rNYOgGtAaYHFcZRhmhlZxSGQb1zMuI6LsZR3tAh9M2IxJ+iMadDzfErJcpvKOSGTyV1/IzY6c8wunrdzg9y8/CWy+YjK7y0sib4qsX19K2a7Hy5/GZtjGUjGdd7ykZQ60Zy0nWhWnaNKJP69mUDmxkSHVs9WphxdzqNrwuMqoZC69PhY/qJHpeYcKatCk3evBbflbfimv5WXmi6UfTs/wsvDoUlJ+cBo2O6Vr0W/EsbdO/5Z/Hp11jWg/4SdO08VMdymG1ESc1yEwjVq8W1jSqXveVNG/hlXacTqLnFSacnu4l/VYDzbSksuiY0jSNWnmiOrTy2z1p+Vl4c1S1yVP4flZelrbiWN55fVo50fRoDTHVv3QsW6WOI9lI04jVa1hPql91NklfvTRv4ZV2nE7svNLRedOgpRf2G0xPpnPZbqt761CL6tDKb+lZfjqu+1HXq3tYI73m5/TSv+Wfx6ddV5qatnXq8kFy8j3IBgJDESjMaTWDoRvQRgtk4GQodOPLIOlf3y2sHAH9VoPFRpH024yZGRQLb8ZUYZR2NL3w+bABVK++wtvogOJbT6umv1h+Zuyi6et3OD0rj4WX0TfnXGF1zcpP3y1tuxYdy+s/E2O497krgumJJzR3eyiVJotUmKbtoahPjeqqTvVvutV3q1eb3iUHT/Wr6U86H9WoaSacdpxOoucVJpyejfJqepbO6R6Sg6kw6iE1jdo9EafDcHqWn8LbmiGbPqP0pW+b6mtp27XofF7/aHo01sZPdWgdZtKBdGsasXq1DiaFU/1ab79pPqwZnQ+nrd9RnUTPK4zytvSUr37biJk0rc4tOZoaPbD89BmXvpXf0rP8FN6mm8vuK641Ku3+tLQVR+fz/Ldypuq0TpCdNt2o/q2BLR1Jt6ZBq1frYLJOWZvOapq3ujCNWdrW0IrqxM4rb2nGNGjp2XpGW+Ol54IaivJ3pEHLT+VU/Gj6Vn5Lz/LTcZtubmt3ze+K03+eelZedl2paro+d1EBM3KSeSuEhsBJ41OY02oGwwyeHDcZQ3NM9cCzh56MkR6u+lfjSA2XlRpiMmJKSyNsNvdavy29aN4yBDqnMPoug2dTBGQYZah0To5reNTKjJ3ihNOPpmf5WXjlpWuRQbQpL4ovo2yG2Yyj0srrPwtjqLus4XrPBI2xu3K46wrTtD0U9alRXdWx6tUcU323erWHoBorGkHSHH9pOqpR04ymDyq+dBl2gi29cN6mF4W39NRY0j2mMqmn36a7WCPRNGr3Y5wOw+lZfgqva9V16F9ls5EEhVdZLG27FitfHp9oejT7YfxUh6ZZ1as6GkwjVq/W+SCdSdMa6VenU1SjpjFblyMtqoMgqpNw3qYV5W3pqTzSnDStBpPZbHuGmEb1GadDK7+lZ/kpvBpySldOsDoxFEZ565mitCxtuxYrXx6fVs40ndZJstPSq+pS9W+a1W/pyTRo9WqdD9KyNG1+gmne6sI0Zg03+SjqIIjqJJy3tGIatPSUv3QnXcs2S28qmz1DLD+Fi9Ohld/Ss/x0XB1uSkvXoHRsVpG91sfStmvJQ8uWh+WdtqabwTr1HGfk5ODikMW4ESjMaZXRk5FQD5NuRs1Z1m+bOqVGivXc6Lx6pGTcZDxklDRqZcZGU1P03dbHqLGkdSoynhZGn5ZeNG+lr3MWVr+Vpo4pP/3rYWzzti2/8BqycPrR9Cw/C6/eVjXw5DjIuKunSvF17Za2XYvSyus/M2O4d/5GvyFWax/OYTvZwjStDgDVo/VmSjP6rYet6lDfw/WqB560JV1LAzoX1ahpRtpTo12alqMZ1Uk0b8vP0tNvlUOOghp9cn6Vn+4VnTON2v0YTT+anuVn4eXQyFlV2tKRrs2u3dK2a1Faef2j6dFYq35Vj6pD1Zn0o9/So2kkXK/SlDQgLWj6lPRnmo9qRp0DarTLDsr+R3USzds0aOnpt9b56L5QGnIozWHVOctPn/odTd/Kb+lZfhZejrQ9c1Q+e0bp2i1t07/Sz+sfTY/GWnqVhlX/qkt9l67DGrF6lX2UhqUvaVXPboU3zUc1I42ZHZR+ojoJ5638TIOWno6pwaSOBdl6+Ql6Npi2LD+7H6PpRzVu+Vl4rXe0Z47KJy3p2nWdlrbp3/LM4zMrTT+y97nLg07gY4/V584ZNwee6xkPAoU5rXnc3OSR7IGVlTFU46vhekdkEJt752/M+NZB0zk6hWW/x9B0MhtQ9vqkfNlM45JNxk5zrxR1f2Vlp6Xrhuu9IN9j1m1PZ+x7kDwEhiKA04rT2u1ty9IYzrrenqBn6pmhlDp4JDSNptE0GuhqoCjnMqt8sdM0mLLSVlHpZqnpRq29fbEh5r00uBtBSAjkRwCnFYel67BkaQzDLw596O7nL8tQ4mgaTaNpNNDVQFHOZVb5YqdpiGWlraLSzVLT9frcuobrHfMbY2xln6HrRdLDEsBpxWHpOixZGkMJtOF6T8oYZrxwFk2jaTSNBroaKMq5zCpf7DQNsay0VVS6mWu6Nv9oMCOnNayzTDwIZEUApxWHpeuwZG0MZ++Z3xIYwywXzqJpNI2m0UBXA0U5l1nli52mIZaVtopKN2tNhzbtOKHZOVk51KQLgWEI4LTisHQdlqyNoQTamPFe9qcIZLdwFk2jaTSNBroaKMq5zCpf7DQNsay0VVS6uWi6u2mHt2cYZ5k4EMiKAE4rDkvXYcnJGN4VjIq9kJGo0TSaRtNooKuBopzLrPLFTtMQy0pbRaWbi6aDTTtObt5xOCPfg2QhMBQB32nVuyWKugHJtzwPFb0vRS98TPulimFlPlSfW99wvRNqjGm6QPhcSt/RNE54156h6fLYF2x9OnWBptPhiB7LwzEPTYc37Wi63nUp+RskA4GRCeyS4/2ut7/3nzFK5TFKRdWFXh4ZNMRuG1lZKyTQcNv7/VGx2vyjKwQb9pSv6auv/MBCURzJtzz30phouqb7Ui/yRlvl0VZRdZGXppu19r4M7TSapsOsa88uuvC7Xgt8j23DPvgHidfobtrR3j9IeMJAIA8CV0n8Z5yx7hv7H5/v3hRFPWDItzgn43f2/UXn7LPOfT0whpdkKb7ZWntzMD3xmHqpUs7L1/S6dWctoOni9FSGe1maPuvMc+wBX2VNb9F9ufGC7/jGH3z5Jez0BDuw0vSZZ579rTGw076mL9z4na+h6cm201+Y/SPrAF5wHCfTjTTCm3Y063Pnp+x7kBwEhibgyah/97+5YuH3vnSIh/yEPuS3/ehHzRgeGlpJCSLOut5Laow1a+0sRt98TX/Pd73rdTQ9uQ/5MdK0OitelZ3e8dG7sdETaqPVuXHT9bcVYqdna+1bE5j3QYJ2NX37T30GTU+opg/8xnPqYLIO4FxGqRrdTTva04MIlTAQyIPARfaQv/TiyzsP1L+MUZwgo6iGyo9/cGdn9eo1Mobqkcpl7nTT9e4MRsXmMhA5mp4gDUdH36RpTeNbtWq1RsPGRdPqsOicdtrp31RjjFGEyepgGGdNn376um+pMYamJ0vT8jVDjbCXHcfJZYTq5GYd2wPf48UMfA+ShMDQBC47uUGDdpLxe9vUIPvRH7y1o008Pv2pz/E/hgzkzGmB7JvXX2C9Uap7ra/K5e+x+tw5Ddc77hvE+vNZTBtD02Oo25XskWn6Teu/3aYjjpumm2ajL9jwtm9d/4EPY6PHXOOm6fVv3KAOBRsNGyc73dW0nHI0Pd7+lulZPmZIz5qFk+mUxLBTs2TTjr3PXRE+x3cIFE1AvREyisdCN0j4ZuH7qQfhuLF4xXGcG/MWYHfTjhnvvozy9jW9atWqf0LTXSdu3LTb63rGVdPbbQYDmkbTGdnNJcnmYKfR9Pj6Fr3ss45rurV2OU57nfgS/cb9sE07mjPz8nn5g0DpCJzjOM71J6f0aFcj3SRj/796tfML67/7TK0rGvtrjVyjXmx4reM4a4tQ4ew981v8EbHs3+sxkZrecOW5fxup70nQ9yRoWo6LNjuYGBstHctOo+n8LXVOdhpNT47/IRutJRC5N8Ds7nmo5l0bTE88Uq/PFeL/WFn4hAAEHMe5ZveFWzbv3nTi3Z+5KLfhccA7jgxgw/WOyCDyXo90FRFounPNZzdmMe0z3cKOUWpoOrvKRNPZsV0pZTS9Ep3RzqHp0fiNElsvdpbvoY6GUdIhLgQgkAKBzbsvfGLz7k2dzbs3asSAvxwJzM7MP+g3xGrtfTlmO/ZZndL0Jo2a8JcjATSdDWw0nQ3XQVJF04NQSh4GTSdnllaMxox33+KoGO8US4sp6UBgKALX1Z21m3dvOqqG2DW7N/7ddfWLChsuH+oCKh6pufe5KxaNoXc8g3eKVZzOcMUPa3rz7k1/PVwqxBqWAJoellzveGFNX737wlxesdG7NJN3Bk2nX+eBpv9BvgeaTp9vvxTRdD9CnIdATgRsasDiiJjfGLs9p6zJJiCQ8TvFJo7zMk0zPTF3DaDpdJGj6XR5DpMamh6GWu84aLo3m7zONFzvUDAjJ4v3meZ1GeQDgWoTCE0N6ASNsZeqfUXVK/2s6+0JRsWerV7py1fiGE0zPTHnakLT6QJH0+nyHCY1ND0Mtd5x0HRvNnmdmXW9WuB7PJNXnuQDAQiECISnu9iImD6v2bNJu0bylxOBz9fnNgbGcOGB+hwbpozAPU7TV+++8G9GSJKoQxBA00NA6xHF1/SeTX8fttFMue0BK8PDaDo9uNL0Nbs3/WNY00xPTI/voCk163MXme/xUH1u/aDxCAcBCKREIDo1IGQU6R1JifGgyTRc79nAIOb2stJBy1alcL00ze6J+dcimk6HOZpOh2MaqaDpNCh2d2q2WTjdT+x0OnyTpNJ0PW/R92izLCUJOMJCIA0CMVMDOld/ZtPrapC99+c3XZ5GHqQxGIGG2749aIi9MFgMQsURiNN00MHA9MQ4YBkeQ9PpwEXT6XBMIxXTtJzXNNKb1DTQdHlqvuF6uwLfY648paIkEJgAAnFTuEIjYh0ZygnAUJpLfKw+d07D9U74BrH+PO++GqJmekzh8ntbr9594X8bIkmijEAATY8AL4ga2Gl/Z7ml9lmvG9nEet7RESdKAU0nwhUbOJiWeCxGz9I0u9zGUsvuoKYkNlxvQb6Hpt9mlxMpQwACSwj0mu4SMo684HkJsex/NFyvJWOoBbTZ5zZ+OfTTNNNe8q9zND0aczQ9Gr8sYqPp0aii6dH4ZRGbKbdZUCVNCPQhsMLUgO58bV7w3Adiyqebe+dv9EfEZryXU056IpIbQNM0cHNWApoeDTiaHo1fFrHR9GhU0fRo/LKI3ay1b5PvwZTbLOiSJgR6ELh6z6Y71dDy//dsnNNI2DW7Nz3VPbZ7473X7Llwukd0DmdAoF6fW9twvaO+Qdz73BUZZDHWSQ6gad6VkrMC0PRowNH0aPyyiI2mR6OKpkfjl0Xs8JRbpidmQZg0IdCHwGLjS2sOLtzeJyinMybQqM0/GoyK3ZdxVmOdPJouT/Wi6XTqAk2nwzGNVNB0GhQdB02nwzGNVBqu96Tve7geOzenAZQ0IJCEAMYwCa1swz5U8671jWFt/pVscxrv1NF0eeoXTadTF2g6HY5ppIKm06BIQywdiumkMlubv8WfjcOOoOkAJRUIJCHAAz4JrezDNlzviG8QmZ44NGw0PTS6TCKi6dGxounRGaaZApoenSaaHp1hWikwPTEtkqQDgSEIYAyHgJZhlNla+wl/VGzGY3rikJzR9JDgMoqGpkcHi6ZHZ5hmCmh6dJpoenSGaaYw63pPyfeYddvsD5AmWNKCQD8CGMN+hPI9P3vP/Ba/IVZrH8435/HJDU2Xqy7R9Oj1gaZHZ5hmCqc0zTTyYbmi6WHJZRPPXliu7eyzyYFUIQCBWAIYw1gshR0M78qltQiFFaTCGaPpclUemh69PtD06AzTTAFNj04TTY/OMM0UQi93XtD3NNMmLQhAYAUCGMMV4BR0yqa9NGfmmwUVodLZounyVR+aHq1O0PRo/LKIjaZHo4qmR+OXRWy9S8yfkeO2b88ifdKEAARiCGAMY6AUfKg77cX1jhRclEpmj6bLV21oerQ6QdOj8csiNpoejSqaHo1fFrEbrrdrcZ2Y91QW6ZMmBCAQQwBjGAOl4EPhaS+NWvuqgotTuezRdPmqDE2PVidoejR+WcRG06NRRdOj8csidqP+/CWLI2LeCe2kmEUepAkBCEQIYAwjQEry06a9NNg9MXGNoOnEyHKJgKaHx4ymh2eXZUw0PTxdND08uyxjNlzvkD8qVpu/Jct8SBsCEAgIYAzLKYXutBde7py4gtB0YmS5REDTw2NG08OzyzImmh6eLpoenl2WMdX5G6wT259lPqQNAQgEBDCG5ZRCeNpLk5c7J6okNJ0IV26B0fTwqNH08OyyjImmh6eLpodnl2VM+RvB9MSj0neWeZE2BCDgOA7GsLwysGkvs65XK28py1cyNF2+OrESoWkjkewTTSfjlWdoND0cbTQ9HLc8YjVc74gaY7xCJw/a5DHxBDCG5ZVAc+/8jUHP1KHylrJ8JUPT5asTKxGaNhLJPtF0Ml55hu5OT3Q97HQC8Gg6Aaycg+rVOfI9ZmfmH8w5a7KDwOQRwBiWt87r9bl1Ddc77jfG6s9fUt6SlqtkaLpc9REuDZoO0xj8O5oenFXeIdH0cMTR9HDc8ojV7VxgjXoeuMlj0glgDMutgIbb3u/3TDE9ceCKQtMDoyokIJpOjh1NJ2eWZww0nZw2mk7OLK8Y4bWP2tI+r3zJBwITSQBjWO5qb9Tmt6khpjfel7uk5Skdmi5PXcSVBE3HUVn5GJpemU/RZ9F08hpA08mZ5Rkj1LmwJ898yQsCE0cAY1juKg9Pe/l8fW5juUtbjtKh6XLUQ69SoOleZHofR9O92ZThDJpOXgtoOjmzPGPM1uZv8ZdFuN5cnvmSFwQmjgDGsPxVPut6Ty0axPbt5S9t8SVE08XXQb8SoOl+hJaeR9NLeZTxF5pOVitoOhmvvEM/Vp87p+F6C/pv1ufOzzt/8oPAxBDAGJa/qpu19m1Bz9Sz5S9t8SVE08XXQb8SoOl+hJaeR9NLeZTxF5pOVitoOhmvIkI3XO9Z+R6ztfatReRPnhCYCAIYw/JX80P1ufXWM6VeqvKXuNgSouli+Q+SO5oehNKpMGj6FIuyfkPTyWoGTSfjVUTok7s23xXMxtlfRP7kCYG8CWjo9zrHcbbn+f/dH1n/1Obdmzrf+SNvfCLPfIO8bnMc5yrHcXh7ex+1Wc+Uel37BC3T6XWO41zrOI5603LTNZoukwR6l6WCmlYniPScm5YtLzTdW0dlOlNBTRdio6VrNF0m5caX5aG7n78smI1zND4ERyFQfQIygtvXrl37ouM4nSL+z7vwjM7Gf3te5+w3n1ZI/rrmNWvWHF+9evWvO46zofpVms0VNFxvV4V6pq4469wzf3ft2jX/OqmaXr1m9f/nOM7jaLr3/VAhTauDbG7VqlXfKkLPyrMMdhpN99aynamQpq9wHGf/qtWrvjbJml61etU/Y6dNvfGfjdr8K8H0xM3xITgKgeoSuG7V6lX/jxnB0884rfO9V17W+aEPvr9zy8dv6tz2Mx8e6/+bP3pD59rrr+5ccOGGbgNw9ZrV33Qch61SYzStHRODnqnj2qErJkgZDp1z3hvO+/Kq1ateM11ffNlFnfff8N6J1fSq1au+gabjpVkBTWuWwtOm5TVr1kyUjdYzKM5Oo+l4PetoBTStUd3Hw50Kk2Sje2p61Sp1GuJ7xEi7UZt/VL5H0/XujTnNIQhUlsAue7jLCNYe+XTn9/9qf+d/fvngRP7/6lO/4jvrxmTN2jW/5zgOa6Ei8m643iG/Z+qe+S2RU2X4ueGc887+P1WH6lRQZ8KX5n5tIvWs+ziq6VWrVv0Oml4u0xJrev3JKbUvS8/nnX/u69N3b+/87l/+1sTqOU7TjuM8iaYrpWnNOPFn32CjF32tqJ1G08v1/PA989cHncAvLD/LEQhUk8C0Hu7qXf3YXT/ZefpvvjLRD/dw4/Nz+9zO2eee7Y+QrVq16k9YO7ZU4OqR8g1ibV5T3sr0d8455531t9L1posv6Pz6HzfQdNCpEtZ08JAvU70VXpYSa1o7lHbUUTbJHQph+2zfI5p+Bju99DYqqabVsXkIGx3f2R3RtDoY+AsIBO/IOyHfQxvSAAYCVSdw0arFIfDOrl+axlmNGQGUE/+G9efbdEVtIMJfQKC597krgp6pI2WCcu63nfOUPeCf/PMvouuIrqVp62Bg+stS5ZZU09ukZ9mhr3hfQM8RPatBhp1equPwr5JqWrve+R1l2Oj4xhh2Oqzipd9tExq2sV/KhV/VJKCels4PTL2Ph3vMw916XB//wwf96W1i5TjO9dWs6mxK3XC9I4ujYm3tNlmGP+0i5zc0fvNPH0bXPXStHldxCjpi2JQmpNwSavol1dXP3f8p9NxDz7LVstOnnX7a69jpkJiDryXTtG+jzzrnzNew0fGNMPM9zE6f7DA7wUZLp3Rt29g3a+19p47yDQLVI6CpASc0JZFe1pWNoYziHTMft1Ex5iWHtG4LZxsz3n2hw4V9PfPsM/9XOWJaBG0PMz7j9a1NSwKn9cHCKqyEGZdM0xepjjSCybTxeB2H72/sdPwNVTJNa/ooNnqFToWwprHTyzUd2sb+1eVnOQKB6hC4Rcbwnde8HYd1AIOozUvO/bZzbPe9y6pTzdmWdPae+S3B9ETN9y/6b712SNTC70nfyCD8IO/1XQvDg4ZYqaaWFi2ikmla77zzd3TtVY8cP9VAk50+65wztTOotI2dDm6mEmlaa3oW1q5d8y1s9CndrnQPY6fjnwiNWvuwfA9NvY0PwVEIlJ/AXXpYaTe5lYwA504Zy2u2vNscV63Z4O8kgXp9bu3JaQLHfYNYn1PvfZF/vtN65fvegaYH6FzQvR16XUNZppYWqR8/75JpuiY7zQjvKTvc75n07ve/U9O4ZKux08HdVCJN+x3A77jm8n/tV4+cP6X5N3/7ensHJnY60PRsrf2E/I5Z12Ob/8KfmhRgWAJ6B0NH2yBj8E4ZvJVY6B02wQN++7DQxzFenpDoNwAAIABJREFUw23vD0bF9BqEIv/oXBiwAWY617vzAk3LQeIvIFAiTWOnE2oaOx1/G5dE09johHqWrd78gau+jp1equvZ2vwtgd8xt/QMvyBQHQL+A54F4IM1wmQM1StNQ2y5wJu19m2BQdQW20X+4bQmfMjjtMbLtWyaxk5jp+OVOvjRkmgaG53QRsv3wE4v13mzPnd+w/UWGq534rH6HO95XY6IIxUgQEMsoUGkIRavahnBwCAuyDjGh8rlKJpG06kIDU0P3vCx0dWyfGKn42+BkmgaG53QRtMJHK9nHW243gvqBNZLnnuH4gwEyksAg5jQIPKA7y1me6+Hel17h8r8DJpG06mJDE1XszGGne59C5RA09johDaahlhvPYdeVv5o71CcgUB5CWAQExpEHvC9xdx0vTuD6Ymt3qEyP4Om0XRqIkPTNMRSE1NJEiqBprHRCW00DbHeN89srb3Z9ztmvJd7h+IMBMpLAIOY0CDSEOst5mZ97qKgIXZcO3T1DpnpGTSNplMTGJqmIZaamEqSUAk0jY1OaKNpiPW+eYLdQI/J95C2e4fkDATKSQCDmNAg0hBbWcgN1zskg6h31qwcMrOzaBpNpyouNF29xhh2euVboGBNY6MT2mgaYn31/KTfCVxrs5v1yqg4W0ICGMSEBpEH/Moqbsx49y0axPmi5mujaTS9skgTnkXTNMQSSqb0wQvWNDY6oY2mIbbyLTXrtqd9v8P1ilwWsXIhOQuBHgQwiAkNIg2xHkoKDjdq7asCg3hk5ZCZnUXTaDpVcaFpGmKpCqoEiRWsaWx0QhtNQ2zlm6ZRf/6SwO84unJIzkKgfAQwiAkNIg2x/iJuuN4RGcXm3ueu6B869RBoGk2nLio0Xa3GGHa6/y1QoKax0QltNA2xAfRcax/2G2O19lX9QxMCAuUhgEFMaBB5wPcXb6M2/7jfEHM96SvvPzSNplPXHJqmIZa6qApOsEBNY6MT2mgaYv1vFtPzrOvt6R+aEBAoDwEMYkKDSEOsv3i1UUcwTeBQ/9Cph0DTaDp1UaFpGmKpi6rgBAvUNDY6oY2mIdb/Zpmtzd8S+B3P9g9NCAiUhwAGMaFBpCHWX7z1+ty6husdl1H8fH1uY/8YqYZA02g6VUEpMTRNQyx1URWcYIGaxkYntNE0xPrfLM363PlBQ+yEtN0/BiEgUA4CGMSEBpGG2GDCbbheKzCKuwaLkVooNI2mUxNTOCE0XZ3GGHY6rNze3wvSNDY6oY2mIdZbw+EzDdd7QX7Hw/fMXx8+zncIlJkABjGhQeQBP5icm7X2bUFDLO9pAmgaTQ8m0oSh0DQNsYSSKX3wgjSNjU5oo2mIDXYrdV/LMOPdP1gMQkGgeAKpGMS3vPVNHfvf9UvTHRkN+7/hx3+oe+57r7yse9zOD/N5y8dv8tOcvnv7wOkprMr4/hveO3CcuLLREBtMtI/V585puN6C/jVlYLBYqYQaWdONg/d1NSvN3PfE3Us0c82Wd3fPj6on05jSUV61Rz69JC87H/dp94E+484PegxND6a7KmtaujIbrc/H//DBJZq57B2Xds+PqifTney98tL9ZMf6fZqmk9j2uDTRdKk1PbKNVp3bM910ffD5J7o6+/2/2t+54MINqWva8vrS3K9184rTX/jYMLY9HN++o+n+mm663nVBB3AR69P7F5AQ409g8+6Nt23eval2zWc3XjLg1aZiEB3H6di/nFQzHPp8w/rzu+dkxMLnhv3+Qx98v5+mDNOgaZgR+773fM/AceLStnQcx+EN7n1E1nC9Z2UU1evaJ2jP00Vo+ld++56uZqXrmz96Q1czv/uXv9VZs2ZN9/yoejKNKR3l9XP3f6qbl53r9Wn3gT57hRnkOJruKb9lJ6qqaenKbLQ+P3bXT3Y18+t/3FhyblQ9meZk75WX7ic71u/TNJ3EtseliaaXSbfngQI0nYrfEapjX2efbezq6ixqw9PStN1D+/7zo9284vQXPjaMbQ/Ht++h68X36KHm8LrHB+pzG3oE63t4CL+jb5oEmBACm3dvvHfz7k0d/V+9+8JDAzTKUjGIZpxOP+O0zplnres8/Tdf8Y3Urz71K76BNMeVhtiECDG4zNDb7p8c9sqL0LQ9xKVnaXvTxRd0H7r1L+xZomkaYsPWbDXjVVXT1hCTfZamr3zfO7qa/pn6J5ZoOi2nlYZYNTRegKZT8TusYWKavv7DP9DV9E986kO+ps03SUvTlh4NsfJqu+F6z6gDeLbWvnXYUob9js27N/31AL70sFkRb9wIRMTjN8j6NMpSMYhmnKznx6aiaOqAzmnaiz6jDbEHv1zvRKd5RafMaLqBDKziyiH+xO6f6sT1mqpX99rrr/bDaUqCpkSGpyqY0R7VcbZ0GBHrf/dox8RgmsDxYXcxKkLT1hCT5syZ/Ir3Bf8hr9GxsKajelJDTU6u4pkOo9NY9BA3rb7t0k2dzzzwMx27d8IjYo88+bluWtL+Rz65raMROeshtftgVCcDTffXsoWoqqatIabpgnJc9a/pW9KStBjWdFRPimvTDKVD3QNh26o01Olmur/08os7ug/s3gmPiH1un9tNS9qX9qzjTumYpnXcdD7MJ5o2xfb/LEDTqfgdVsfSXdS/MJ/j4ssu8s+FNS293THzcd8vkUZNh2HbKs3Jj3nnNW/3dSz967fy0X+4IaaROJvaK+0r7bBm42x7+Pyg3+168T1W1vTJHZvvCmbi7Fs5ZO+zQ/gdvRPjTDYE3vuZt16xefeF28v2f83uTU/ZiFivz8hIWSoG0YyTGYqP7rrVN0RmIOVARg3lPb/6890pXue94dzO2eee7YeRg2ANORlMM6QanZBBs1E3paf8ZMRkFK1XTI6CHGCd13dzNqxsUcd5UCNo4ZTO2W8+rfOdP/LGJ8pW/2Usz+fcp/8vGcUdd3/2kWHKV4Smww0xcwytgaSHtrRlmg7rSQ9gndO/puSaJvX9N//0YV+rT/75FzvrN7zRDyPN6wGvEWMLa/moEWYjcroHLE44Pytb2MkwnSb5RNPJbHkWmr76M5teD9vstO20NcSkH7PL1kCSDqVBrc+SdsN6Mp3ruDRompRttcZY2P5K63JKlZ6FtXzUOLPZEQpj09Z/YOp9XcfVNG22PYmOw2HRdHk1fd6FZzwiPZmtC9dbku+qY9Or9KjvsrOysdKZdG16Cmtaa7YUVv9qiJkmZYvNX1CHsOlXupcNtt+KZw2x8Giy4psd131j16J7TnHSuN5zN56B79HH9771sz9Rk8/xgPuf/2EYn0NxhvA7smlskGpvAr1ay+EHaRW+v+MTG/523RvWjmwgzKiZAyvDo0aUjJKMnE3nktGTcZKxs4ewelcVVv8aHVNaMnoKZ/FkADXipWP6NINoD2s9yBXPNk5QWurJ0jEZSsUzox12ZHU86b/S2fhvz+uONlahnoss48fu/tmOjOLP3P2LuTBLQ9OmY+nVHFg9yDUqJk1Ju3bc9KRzpktbfyOHwHpmbe2kNdbU+WCjbJaf0raHtT287YGu3lprBOq+kG7jnIykelZ4NL04nXvQ+yRvTW+48ty/DWtjmDoO61X6VHqavqUGv75Lp2YjzWmVM6pz+rf1N2p8mdOrWQcqizXW1AFmowqWn+JaQ8w6yMwmKy17DmhETWmZplWWYa7T4qDpcmv6u299U+eOX/7oyHUsfUkz0qK+S1vq5NV32VzTk2k67FOYLtWoso4u60Q2nyLcOLP7Q2krjvwY60BWntKejus5IL/HOirMlpttN40m/VT+33nDG3N5jg5qC8sa7nO1P/L9jh/afXU+vPZsejbB3gy9GxecGZzA5s9uvG6xMaY1WSX637Nxru+NsWfT32/efeET1+y+cMvqtc4vyKiMaiCUhv5lmGSE9K+d5nRMD3hzNK0hZr9lrKwHSkZJvVmWlgyaGT5zds1wmWHTeR1Tuoonh0Bp63/rbdf7xzTtRmF6pWVpDvqpdM678IzOd39k/VOlqvsy6TBUlq2f/eFHgt6pr12z5+JfSMysAE2bPqUr6VDaksMoZ1Tf1eA3R9O0aefUwAprSVOxFEdaVweBOQbhUQCFNyfV7kWFVzxN7zVNW0+ubR5iaZmTEc43yXc0ncyG56Lp3ZuOdu306tHtdFivNsVKttkaZbKdZiNNT3ZO063CerK05LzquHV6WaeBjknrNjog/Wp6rvSsf90rpmmz5bZLomlaZQnnmfQ7mi6vpje889zfXrV6dL8jrFezv3re27NfmjI9mabtnPkFpis1wKRN07rZY+s0UDg1rEzDei7YfaRjpmd9WoeZ7YBrGjfbbnkm/dT1vuHSM/E9Qv5FL3+itnffX8vvuPPue/+4V5gVjw/gd1yze9Mxs9HX1Z21g7cgCDnWBBaFFdsTpiFav/EVEUyqUxNlWMzo2EiANY5krKwhZkbTHuRmkPTwNkOn9WO2HidqNM242sPa4sR9mqNsRtt+W55JPy0d5mkPfivNut5LMoqz98xvGTzWYsgiNK2HaViv1tA3TevhbM6o6cmcVvWghjVlDTmlJ2fUtGuNKQtr943SDceJ07Q5FZaW/ba0kn6i6aSqdJxMNB3qJEvbTof1ao0kNfbleEpj6jgzHZie7LeN5pqu7P5QPB0z7Sq8hdGn3TcKbxs3xelZxyyuadp+h9NL8t3Kjp0eXNs5ajoVv8PqWJqx2QoaobIRW2nO9GSajv42Tdn9YT6KaVfHLYw+Tb+y0eH7wI6HPy2u3R/2O5xeku92vWi6v6YbtfZ2+RwN13u6f+jlIXr6HXsu/Odr9mz6ogYyIjZ6eSIcmUwCYfEM2FpPxSCa8ZFRCRkL32jJWJnBMiNn02EUz4bvFTd63Jxb66Uyo2XOg/LSMeu9sh4oHZOzYeH1aeUyxzl8Lsl3SwdjOPg9Zi9ZnK21nxg81mLIiKb/sUeHQjjZkTUd1as9vE3nmh5rD27Tk7Sn83IEwtqz4xodkM5sCk20c0EjaYpvD2ub5qh7wvQZTlfHrFzmZFi4pJ9oOiyfwb6npenNuzf16iQLF2RkTUf1auvEpDk1yDSl0HRgetLGSDovxzasKTuunn8df8917/LDabTBwik9G9XV/WSOstILb17TS9Mqi6U1zKddC3Y6LKOVv+eo6ZH1LE1YHZterQEmjckOK0zURlrnrkZxw7qy9ZF23NamS+sWLtxBpu9hfyWs4/B3xaUhtrLusjirreuDhpg2Cks8WrXE79iz6fg1ey787QIaXw+etF+HQ/+tGFY6Fg6jONG/8Hl93xUNwO8UCVy9Z9OdAziq4RxTMYgyfPqX0TEnVr/lTGrqoR2zhpgMlTWe9BCXUdN6BBtxkJOgtMKGzl4UrU/Lzx7W5txqNELrcpSnGmtybi2eGW1znM24Jv20dHjAh2W08vfm3ueuCIzi0ZVDLj9bhKajejUnVrrTFEVpxo6ZnqQ7Wy+gXT7leGrqiunc1tNYw0xOqkYhdC/YtBilbw0xc5Q1HVFhlL5GkJW/vWA66mQk1bKFR9PLddfvSNU0HdWrdXJJc7K70oLpwBxb2WVrTGk9mTStqba2rktpKJ6te5T+ZbNlf80mK33dTwpnU7bUYJOm1SBTB4WeC1msEVPe2Ol+Sj51PkdNp+J3RPUa1pyN4kZtpGbaBLrw15PJrmrdmE2jNftrDTNpU0sm1LGgNC2u7g1p2O4FW1smHeueUaNQYaR7GmKnNJbnNxvh1Uuek+Y7hN+RNItBwmvXx67mHMeZi4mkY+EwcTtFhs/ru+4//kpEIBWDaEKQ0dFD2HrzzUmNOrYKJ4Noxs/i61MGTA98hdG/nNrweX233ioZYoVReFtsq7wtXaVlI25mtK1Mln7ST0uHB3wyFTdq86+oMfZQzbs2WczEoUfWdFSveqCaBm1DmKhjKx2pkWWOq4U3vdomBgpna2osjD249dscAXVMWMNOn6ZpdTDoHlM6UScjqZYtPJpOrDE/QpU0HdVreH2Lre0yHVhDTPrQOps4TaujQI6owljHl+lZn9KrnFh9t4ZY2OZL05aujUIoLdO0ymL6HObTrgU7nUzbOWl6ZBstTVgdm15tyYM0F11zaGEUT3oPa9W+m21XGPkN4RE2hVGjy2yyNbKiuz+bpsNrgGmIJdNgWqFnZ+Yf9DuAZ7z70koz53RoiOUMvKjsUjOI4Qen1tDot/Xcy2jptxlHe7CqAaWeVj3UNTKmHlb1UNl5+9Sols7LUOphrge70rMHvBlOS0vp6bs1wnTe4pija2kn/VS+geHm7fYJVGvTXpoz880E0YYJOrKm4/QqbarupT9pRj2f+h3Vk3pP1Zsq51Ka1WiBNZxMa3JglZ7C6IGtUQQ90JWejQworMphaUnTihNu0Fkc27HL0k/6iaaHkZnjVEnTUb1Kg6p3/avRL82YjYzqSXE1iiWHUlNq9d47a4SZ1qRLjQpI0xqZ0H2ge0Ppm9OqsJrWK00rLY0w6P4Ip2WaDtt2yyPJJ5outaZHttHSQlSvet6bpm36q+kpqmnFVSevdCi/Inpe6Ss9NdoURlMalaZ8GOUR9lNkv6V507Tuj7BW7T4I2/bw+UG/o+lkmn74nvnrg5k4LySLWZrQNMRKUxXZFiQVgzioIRmHcBjD4QQZmvZyZLgUBo6FphO+lgFND6ytJQHRdPLXf+T1DEDTS6Q68I9GrX1V4Lxmaaex0QlttO4bND2wjP2A9frcuobrnZCem/W585PFLkXobSc7/XWv2H9c57+O2Xl9Kk70L3xe3xNP1YwmyO90CahSlvXq5/WwrGI+GMPhBdhwvSMyihlPT0TTCR/yaBpNV9EWr1RmNF1qTWOjE9poaR1NJ9d0w/Welc8xW2vfmjw2MSCQDwEMYkKDiDEcXpialuj3TmU7PRFNo+nhRZowJpou56gYdjqhkEPBc9A0NjqhjaYhFhJogq+zrrfH9zlq7bhNLBKkRFAIZEcAg5jQIPKAH16MGgmTUTz5jg9toZrVH5pG01lpa1m6aJqG2DJRVPzAKU3Pv5LRpWCjE9poGmLDKfGRvc9d7vscrpflVNvhCkcsCAQEMIgJDSINsdHunRymJ6JpND2aSBPGRtPla4xhpxOKOBI8Y01joxPaaBpiEYEm+NlwvVfVGHvo7ucvSxCNoBDIjQAGMaFB5AE/mjZtS9kMd09E02h6NJEmjI2maYgllEzpg2esaWx0QhtNQ2z4W6ZZa+8LRsXuGj4VYkIgOwIYxIQGkYbYaGIM7TT36jBvvB8gdzSNpgeQSXpB0DQNsfTUVI6UMtY0NjqhjaYhNvx9oY06gobYs8OnQkwIZEcAg5jQINIQG12M9tLQ2Xvmt4ye2rIU0DSaXiaKrA+g6XI1xrDToys+Q01joxPaaBpiw+tZW9c3XG9BW9k/Vp87Z/iUiAmBbAhgEBMaRB7wowux6Xr3qodqttZ+YvTUlqWAptH0MlFkfQBN0xDLWmN5p5+hprHRCW00DbHR1N90PU8+R3Pv/I2jpURsCKRPAIOY0CDSEBtdhI3685cEUwWO6aWLo6e4JAU0jaaXCCKPH2iahlgeOsszjww1jY1OaKNpiI2m/FnXq/k+R23+8dFSIjYE0ieAQUxoEGmIpSPChusdCgxj3JvgR8kETaPpUfQzdFw0XZ7GGHZ6aBkviZiRprHRCW00DbElskz8o1FrX7Xob2T66pzE5SICBEQAg5jQIPKAT+fGabjeXcGoWCudFLupoGk03RVDnl/QNA2xPPWWR14ZaRobndBG0xAbXe22jb3eLTZ6aqQAgfQIYBATGkQaYumI7/P1uY1BQyztBbRoGk2nI9KEqaBpGmIJJVP64BlpGhud0EbTEBv9VtGadPkcs663Z/TUSAEC6RHwDeIdMx/v6Ebnvz+DWz5+U+fkSKL+t6dXDZOZUsP15hYbY+3bUySAphPey2g6PfWh6f42NI/nDJoutaax0QlttO6Zm7ff8K/4HsPrerY2f4v8DW3cMXwqxIRA+gTUM9D5yCe30Qgb0DC+/4b3WkPs1vSrY7JSbNbatwWjYi+keOVoekAtm0OMptNTX0aa9h1XjcZbnfG5coPvfT+yeSFwWrHTI8o7A037NvqWj2/9BjpeWcdhPpt/4Kp/RtPDi1lb12sLe21l/1B9bv3wKRETAukSuE039jVb3s0DfkDn9eLL3vZ6YAyvTbcqJi817ZjYcL1jaow9dPfzl6VEAE0PqGV7yL/t32wypxVNjyjCjDSt0ffOD0y9Dzs9oLY3fscFX8NOjyjmIHqg6eMp2mnfRr/zvW8/ZjaIz/4NsgsufMs/oenRNN1wvWf8zt9amxlNo6EkdooENurGPvOsdZ3f/6v9POT7POS/NPdrNhp23HGctLddT7Faq5NUozb/uAzj7Mz8gymVGk330XHY6UHTKakulEwGmr5Cdvotb30TNnoAbaPpkBhT+pqypn0bvfa0td/A7+jfAJO9RtPpCLnhtm/3/Q3XeyqdFEkFAukQ0HzZDuvE+hvE0LqD/emgJ5XZWnuz30Pleq/W63Nr0yCyas2q59B0fz3rAY+m01Dc0jQy0fTqVf9Dmn7kyc/RGOvTGPvQx256Tawcx8FOL5Xm0L8y0LTvd3zK/dhr4Y4hvsfb7dD6MDQ9tIodJ7z5TAbvMB2hZESddAJ603jnDevP73zF+wIP+R4P+V//40Zn7Wlr9YDXNC71UPOXEoFZ13tpcbrAfFrvFEPTPXQcdnR8Ta9d8y00nZKQQ8lkoGl/ndh7rnsXNnoFbUvTa9askY3GTof0mMbXlDXt2+gzzz7zX/A74htfZqvRdBrqPZVGw/VeSNnfOJU43yAwAoFn1Bj73isv6zz9N1/hQR950D/551/sXPC2DXJY1cvKm9lHEFpc1NC7ap6OOz/kMTQd0bE92PW5qOm3fBNND6muPtEy0PQGx3E0Jbrz2cYubHSMtqXpDRvffAJN9xHnkKcz0LRvoy+9/Dv+Bb8jvjEmTb/529f/C5oeUrQx0WZdr+ZPT6y1n4g5zSEIFEZAD/nDutnVGKOH6pRR/M0/fbhzwYUbbDODFx3HOaewWhrTjLWDkXYy0v8D9TlpMY2/DatWrfrvaPqUlq0h5mv6bd1GGJpOQ22RNDLStL9px+nrTv9W/Qt7aIyFGmPSdKgRhqYjekzjZwaa7vodl15+8dfwO5baamn6zRe8yTadQdNpiPikA6eNwYLlEEfTWg6RUtFIBgKOdq3zG2Pn/0/nvaY1Y5O8kPZ3//K3Otou+syz1lkj7JDjOGk1EpBbhEDD9Z6UcWy6nqZgpfUX0vS3vY6mFzW97swzbCQMTaeltJh0MtJ0U50Lq1avem3qp35kQT3m1sCexE/fTt/54dfOWHfGN4JRAzQdo8W0DmWg6a6NPvvcs76uNWOT7HfoHpamf/LOWxZOX3e6vTcMTacl4CCdRm3+FX9U7J75LSknTXIQGJmAGhr+dAE91NTzqq3t9Z4xObE/d/+nxvp/+u7t/uYF73zv27+5dnHtjO2SqAWyjISNLK/eCTxU864Neqm0aUeaO1Iu0fQZ6854rShN//z9P5v7/WOavuKay7+BpnvrL4szGWr6vmANlL8e6opr3v4vRdnoIjX9jqsv/3qwHgw7nYWAY9LMSNNLbPRpp5/2zXd//zu/Poma/r73fO/X0HSM8FI+1Jjx7vMbYkxPTJksyaVJQAtp/V2Ngl5Ge9BN0qdGwtQo5d1KaSprhbQarnfIb4xl846PQjV96Xdc2bny8h8u+v5B0yvoL4tTGWpaGwbJPtmIfe7aQtNZKKb8aZqm9aLnlEtbqI2Wr4OmU67RkibH9MSSVgzFiiWg933I2O45aaQ0ZSyv/18IGoJ55RfO5y7HcW51HIc3r8dKIruDerAHo2KaipHVX+6a3vDm73jw/t3/6fjWLXfo3SVhreX1HU1npaY+6eagadkp2atcbTSa7lPxY3w6B03nbqNll9H0GIs25tKsQ2GW6YkxdDgEAcfRvF3tfsWarAlSg6YkNlzvVTXGmq533ThcuhYDN1zv2QxH+sYB09heQ1jTmtY1DheKpsehFoe/BjQ9PDtilocAuyeWpy4oSTkJaFtRTbXRyBh/E0SgaxzH5M332nwkGOXrNLKZcjlB6qjmpaLpatYbpe5NwOzaLHa6NyTOlJpAo/78JcGzWbsnprkuvdTXTeEgMAiBtY7jHA0aYv/oOA43yCDUxiSMtq9vuN4JbWXfrM9dVOXL0pSHYFv+DiNiVa7J0cqOpkfjR+zyEUDT5asTSpScQMP1XtSzebY2f0vy2OWIofVuVS5/OShSiigBTUsMLzy/PRqA3+NNYLbWfkLGsTkzr626K/kXOCr+NMug140RsUrWZDqFRtPpcCSV8hBo1tr7xsFON932/9u10a6HnS6PxDIvSdP17gzq/unMM8sgA9/PqLUPM9smA7gTnqRNS7TG2P8x4Twm7vIf2fvc5cFI0onP1+e0cLtSf0vW0OjBbv9MTaxUPaZZ2LCm9fBMM+080kLTeVCuVh5oulr1RWmXEwheUu7PwKmaXX6sPndOc8b733z/At9ieeVyZGgC4WmJ1hDT5/VDp0jEShLo9rbW2vuqdgG2fqLbAKMhVrUqzKS8aDoTrCRaIAE0XSB8sk6FgL2kvOF62l24Mn+hcjOKW5laq0ZBo9MSrTGm9+XwN0EEgoW0C/7IWP35S6py6VoX1qx531rWCGPKS1WqMLNyounM0JJwQQSqrOmG234NO12QcEqUbXPv/I3SwazrvVSiYq1YlMaMd/8S7TIitiIvTiYjEJ2WaA0xfV6eLClCV51A1XpbNbWh6Xp/t8RA2mgYDbGqyzGV8ldS07X236PpVKp/LBOpoqYbbvsf0PRYyjHxRQXTrv313LO19ubECeQcoeG2b1+mXRpiOdfC+GanaYl/H9moI9wQUyONvwki4C9EDXZQVM9rmS89Zg3N8t5WjGXcwym0AAAR6klEQVSZqzCXslVc06/jAOQik0plonW8ttNtBe30qTW81mmGna6U/tIobGPGuy+wba000ssqjWU7MaPZrFBPbLq9piVaY4wXPE+gNLRzogxk2d9X4zfEavPbGm57/8m55seXOayMiE2geuMvuZKarnnfRNPx9clRx0HTqKDKBIIOMn8pRFk3CNM29fgWVVZZNcq+0rREa4zdW41LoZRpEQiNIHSarnddWulmmc6vzLS/t1Gb/1awVuxfug4sPa1ZYq9M2lXTtEY5Tr6UWk7KNxq19qElzgCarozusiwoms6SLmnnQaDhei3/WT3j3Z9Hfkny8O8vbVNvI2DRT+xwEpyEXYHAnSenJaqhpf+5YIriU6FjOj69QnxOjSkB7WbkG6Ba+3C9PlfqF3z7C38Dg6leYpW3EYyU8dLFMRXoEJdVFU1ri+SG672g+0961qWi6SEqfAKioOkJqOQxvkStDwsaOkfL5GeEbTANsTEWYAkvTY0ujYJtL2HZKFLOBIL1Vy/KCM3OzD+Yc/YDZdesz51vi9YDYzmnd5QMFJlAE0egCpr2e2FnvJd9Pc94L6PniZNpogtG04lwEbiEBBqut+hnuO3SdPpbx1fTbX+5UfO+HtsYY0SshGqqfpFoiFW/DlO9gube564IXvK80Ki1r0o18RETC3qsfAPecL1jsyUy4iNeGtEzJFAZTc94L6tRliEKkh4TAmh6TCpyQi9jcfaKpw1cXi3TqJhVR70+d35jZv6I3xgLr9ulIWaI+EyRAA2xFGGOS1Khd2ccUu9rGa5L5Wi6nmejBmVd6FsGVpRhOYEqaJpG2PJ640hvAmi6NxvOlJ9Aw/W0DlaNsV1lK22wju1YY8Z7+Zd+6X95K8seylZD41UeGmLjVZ+pXI0/RB9MlWq6njRS6F8wFefJwGi/2qzPXVRogci8cgTQdOWqjAL3IYCm+wDidKkJlHVUrFFrbw98jWN0jpVaQmNTOBpiY1OV6V7IQzXv2mCKYqdZa9+WbuqDpxZthGlr2cFjExICpwig6VMs+DYeBND0eNTjpF5FaFTsrjIwCN7Vd8xviDENsQxVMhFloCE2EdU83EVqDVbQM7SgB/5wqQwfS71R3emIrveqdlsaPjViQsBx0DQqGDcCIU2fKIOdLqIM41ank3I9elWOjT4Vvdwgsmvik5NSB1xn8QRoiBVfB6UugXZPDAzlq3rPUV6Fffie+eu1kDfI+wgjYXmRH/980PT41/GkXSGanrQaH5/rbbieLTsorPETzLx52vc32Ll2fMRVkSuhIVaRiiqymF1DmcOublr3oPcoBQ0wLeR9lnnaRdb+eOaNpsezXif1qpZM4cZOT6oMKnndS6cDzm/L+yKW3Duu9yqdvnnXAPnREEMDfQlEhuwzmyLoL9499Vb7Bb24tG/hCACBIQhENH0kq2mvaHqIyiHKUATQ9FDYiFQCAqHptUfyfI9icM88a7N+aISVQAwTWAQaYhNY6cNcskalNDoVGKyFpuvdOUw6cXGCd+I8ExoFe1HH4sJyDAJpEUDTaZEknbIQQNNlqQnKkZRAaD34sxqlSho/afjgXnmBRlhScoRPmwANsbSJjnF6/hD+jHf/qQZTe7+M2bCXrIW6s6731Kn0Fl/SnIcRHrbMxBsvAmh6vOqTq3EcNI0KqkggaBgtrguvzT+a5TVoBkTD9YIXNs+/wkhYlrRJux8BGmL9CHF+GYHg/R/HgwbUiUZt/vFBN/JQuFnXq4W2rdU6sONaF5bnlIRlF8WBiSaApie6+sfy4tH0WFbrWF9Uo9a+Sv5A4Ftk8qJnzeaxV/M0XG9ulM7ksa4MLi43AjTEckM9XhkFDSqNZmktlxpTnUZt/pVgE4S7/Jci6sWItfb2YEcv7Ui02ANl4V3vqF4Y3azPnT9edLiaKhJA01WsNcq8EgE0vRIdzpWRgN5baj6FfIe0Zsg8sve5y0PTHztppl1GjpSpOgRoiFWnrkpZUj3oG7X5R0O9WIuNslONrejvo7O19hOz98xvScvAlhIMhaosATRd2aqj4D0IoOkeYDhcSgJBY8zv5NXyhVE6azXiFezEbJ3Gr2q0uJQXTqEmkgANsYms9mwu2u9xUm9Wbf7RZq29z/79qYi1+W1yBrLJmVQhkA0BNJ0NV1ItjgCaLo49OQ9OQC8Gb7je0WB07Nis6+3R620GTUHTHOWDNFzvRJDGghpkozTqBs2bcBBIQoCGWBJahIUABCAAAQhAAAIQyJyAP5LreuEdldUwa2nJgzbYaNbnLtIW9Br10gYcGknTjJvIMoiFhtvez07MmVcXGQxJ4DrHcdQYY6vwIQESDQIQgAAEIAABCEAgGwLaYbnherbVfHS5Q/zvWvuw1oGpsZZNqUgVAhCAAAQgAAEIQAACEIDABBDQCJle/uy/9kabg9Xah4P16RopU0OtpaUQmn47ATi4RAhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEBgcAIXOY5zXQr/Soc/CEAAAhCAAAQgAAEIQAACEBiAwOOO43RS+J8eIC+CQAACEIAABCAAAQhAAAIQgIDjOIdSaISpIXcFNCEAAQhAAAIQgAAEIAABCECgP4F1juMspNAQO+E4ztr+2RECAhCAAAQgAAEIQAACEIAABNafbITd2+P/hZgGWrNH2DtBCQEIQAACEIAABCAAAQhAAAKjE3g20hA7PnqSpAABCEAAAhCAAAQgAAEIQAACKxE4FmmIza0UmHMQKDOBHVsPXDa99eC2HVMHtu+cOnjb9FTrqvp1c0ynLXOlUbaBCXzy5q9cIV1L3ztuOnAt2h4YHQFLQuATH/ryxh1TrRt9DU8d2D69rXWrdI2WS1JBFAMCEMiVwGWRRpg243gw1xKQGQRGJPDT1+1bNz11cNf0VOvw9FSrE/P/6s5tB9jtc0TORM+ewPRU6/GIfvfLQZV+p6daRyLnOjumWq+o8yH7kpEDBIYnENLwy1ENh34f3THVulf2fPiciAkBCECgWgRui2mI3VKtS6C0k0zgjqmvXjK9rfVS6GEe1xALjh3U2kf+IFBaAtNTBw+Ftbxz64H7p6daXvhYzPcjP33zvvNLe1EUbKIJ3PFjXz1n59SBZ2N028tWezTGJloyXDwEJoqAHNPou8V4YfNESaC6F3vHDV/dMD3VejXmAX90eqqlntcT0XOa1lXdK6bk40xgcWS3tRDVbOi3RgxeCf0+5chubd0+zmy4tuoSmN7W2hfRrDT+wvTW1tz0VOvF6anlmlcHRHWvmJJDAAIQGJxAdMfEo4NHJSQEiiUQPMhPOaNTreNab2Cl0ijB9FTr6YgT8LKd5xMCZSLwqa0HN0e0ato+onWPVtZg7aOdCz4Z7TU+fJaHgNaDRTR9QuvBwiX0ZzUsG/U9eIw1Y2FKfIcABMaRgDYw0HvBwiNiT4/jhXJN40dADa7IA76zc9tXro9eadAYOx4Oy5qaKCV+l4FAsM4x0sBqHdbIb7R8YT3rOyMIUUL8LgOBnVsP3BLR6tG4BtZiY+xgU2sh1SGh6YxlKD9lgAAEIJAlAfVKhRth+q53jfEHgdITCKa0dJ1WrUHoVejoGhs5B73CchwCRRGYnmrtjzitnZ03t66LlifoXOhq32+IsRlNFBO/S0AgpiHW8WcybD24jXVgJaggigABCBRKQC9njjbEbiy0RGQOgQEIBGvDljiiO7a1er5sPLpGQdsmD5ANQSCQK4FgXWNY15o6vuxv+qbWlmiDTa9pWBaQAxAomEAwNbHXuscT09sOPqWptoyAFVxRZA8BCBRC4ImYhtj6QkpCphBIQEDrZWIc0bADu+J3GmIJYBM0FwJxo1zTW1t74jLfMXXgroj+TzC6EEeKY2UgoC3pI3qNsc8Hj2l6LQ2yMtQYZYAABPIi8FKkIXY4r4zJBwKjEAjeqRTzMI99h9iycHqZ6Cj5ExcCaROIG+XSWpm4fKanWk9GHNsX48JxDAJlIaDOr7j34EV0LFvt0RgrS61RDghAIEsCWggbnZb4ZJYZkjYE0iKgkYLIA1y7Je4b9F/TZdIqC+lAIA0CO7cdrEU0fSJuUwPlFXVod0y1Hk2jDKQBgSwJ+C91Xty8QzvZ9pqu2Jne1nowy3KQNgQgAIEyELg2piF2VxkKRhkg0I9AzPbdjOb2g8b5UhPQWplIQyx2lGvH1gPrI+E6TLUtddVSuBgCvo63tm6PvsA80DavGIlhxiEIQGC8CKjRFR0RW7Y713hdMlczLgT0HpplzujWA7HrG7WezLZE1jqccWHAdYwXgego1/RU/HvBNK02qv2dNx68fLxocDVVJ7Dz5tZFep1I8EqGx7VOrNc1RXe1nZ5q0bHWCxbHIQCBsSGgaYjRhhjv7Rib6h3/C9HDOuyQ7tx6UJvPLPnbcdOBa6f1kuepU2vH2Lp+CSJ+lIBAzEtvOxr1jSva9FTrvrCepe+4cByDQJEEYl7FsBD3KgZNV0zyKpIir4m8IQABCKRJQD1O4YaYNu7gDwKVITDtT2s51cAKnNOn/RGDm1pbtANXtBGmntfKXCAFnRgCcbuA9hrlmp5qaX1NeAMaND0xSqnOhQazFqLrwI7LLmukTI2yYAOPFyJ61svJec9jdaqakkIAAkMQ0BSucCNM35eNJgyRLlEgkCuBHVMHWtGH+Aq/X9b7x3ItIJlBYAACy0e5Dh7rFe3kVK9jSzTOxga9UHG8YALBtMRwp0Hf77LpBReb7CEAAQhkTkBbd0cbYj1fhpt5acgAAkMS0LQW7Ri3xDFdOlpgD/6naYQNCZlomRPYOXXg2bCG9Tsu0zumvnpJOJy+M3oQR4pjZSEQzFxYMj08quHg94LWRfbaKbQs10M5IAABCKRB4HrHcfZF/i9LI2HSgEARBIJpMI9Pb2u9FIwYaDv7l7SdvdaJFVEm8oTAoATkgIZfvdBrfZiv88hrGngVw6CUCVcUgcWdPg/u0s6gO6Zar4RGdY9MT7Ve8KcrsuFMUdVDvhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCEAAAhCAAAQgAAEIQAACEIAABCAAAQhAAAIQgAAEIAABCFSAwP8PA/nQlWGl4s4AAAAASUVORK5CYII=)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "afVs3q7tGFPO"
},
"source": [
"Функция для генерации последовательности символов (текста)"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"id": "meZeWXwUGFss"
},
"outputs": [],
"source": [
"def generate_sample(model, char2id, id2char, seed_phrase=' ', strategy=\"greedy\", max_length=100, temperature=1.0):\n",
" \"\"\"\n",
" model - нейросеть\n",
" char2id - словарь преобразования букв в их индексы\n",
" id2char - словарь преобразования индексов в буквы\n",
" seed_phrase - начальная фраза для генерации\n",
" strategy - стратегия генерации (жадная \"greedy\" или сэмплирование \"sample\")\n",
" max_length - максимальная длина сгенирированного текста\n",
" temperature - ???\n",
" \"\"\"\n",
"\n",
" x_sequence = [char2id[token] for token in seed_phrase] # кодируем начальную фразу\n",
" x_sequence = torch.tensor([x_sequence], dtype=torch.int64) # создаем тензор\n",
" hid_state = None # задаем тензор скрытого состояния h_n, при такой подачи вектор заполнится нулями\n",
"\n",
" with torch.no_grad(): # отключаем подсчет градиентов, поскольку сеть уже обучена и не нужно проводить обратное распространение ошибки\n",
" for i in range(len(seed_phrase) - 1): # подаем номер буквы и hid_state в цикле\n",
" _, hid_state = model(x_sequence[:, i].unsqueeze(0), hid_state)\n",
"\n",
" # начинаем генерацию\n",
" for _ in range(max_length - len(seed_phrase)):\n",
"\n",
" logp_next, hid_state = model(x_sequence[:, -1].unsqueeze(0), hid_state) # подаем последнюю букву из фразы\n",
" p_next = F.softmax(logp_next / temperature, dim=-1).data.numpy()[0] # нормируем выходы модели на температуру и применяем софтмакс\n",
"\n",
" if strategy == \"greedy\": next_ix = p_next.argmax() #берем токен с максимальной вероятностью\n",
" elif strategy == \"sample\": next_ix = np.random.choice(len(id2char), p=p_next[0]) #получаем следующий токен сэмплированием с вероятностями\n",
" else: raise ValueError('Хулиган, не делай так! Выбери \"greedy\" или \"sample\"')\n",
"\n",
" if id2char[next_ix] == \"EOS\": break # если получили токен EOS, то прекращаем генерацию\n",
" else:\n",
" next_ix = torch.tensor([[next_ix]], dtype=torch.int64) # создаем тензор следующий буквы\n",
" x_sequence = torch.cat([x_sequence, next_ix], dim=1) # добавляем предсказанный токен в конец последовательности\n",
"\n",
" return ''.join([id2char[ix] for ix in x_sequence.data.numpy()[0]]) # возвращаем декодированную строку"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IJohkcbJGG5O"
},
"source": [
"Попробуем что-нибудь сгенерировать:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b2hqsv33P2gW"
},
"source": [
"### Вопрос 7\n",
"Выполните следующую ячейку несколько раз с одной и той же SEED_PHRASE, запомните выводы модели и объясните результат - чем отличается стратегия greedy от sample?"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5bqp2xRjGIdW",
"outputId": "431d70ec-9bca-4765-aae6-3ae6e143fb11"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Я люблю машинное обучение!\n",
"ддZгàuгuuZsàCCCCОааа___IIшCCОаа!___IICйCCОаа!___IICйCCОаа!___IICйCCОаа!__\n",
"\n",
"Я люблю машинное обучение!\n",
"<eâSайб«.<К»èôOг:cq<ыэôuqьW!Пто'ryS«щаâЖmХàrûЯзVЛОZhJripиDUhLСасм…шп-ЙpCà\n"
]
}
],
"source": [
"SEED_PHRASE = 'Я люблю машинное обучение!\\n' # Ну или что-то другое\n",
"print(generate_sample(model, char2id, id2char, seed_phrase=SEED_PHRASE, strategy=\"greedy\", max_length=100))\n",
"print()\n",
"print(generate_sample(model, char2id, id2char, seed_phrase=SEED_PHRASE, strategy=\"sample\", max_length=100))\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zXrGUveMGKGV"
},
"source": [
"Разделим данные на тренировачные и тестовые, подготовим Dataloader для подачи в нейросеть"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"id": "KNmsiFeXGLI4"
},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split\n",
"\n",
"batch = 128 # Тексты, для уменьшения накладных расходов, будем подавать на вход модели \"батчами\" - блоками по 128 документов\n",
"\n",
"train, test = train_test_split(data, test_size=0.2, random_state=42, shuffle=True)\n",
"\n",
"dataset_train = TensorDataset(torch.tensor(to_matrix(train, char2id, max_len=MAXLEN)))\n",
"dataloader_train = DataLoader(dataset_train, batch_size=batch, shuffle=True)\n",
"\n",
"dataset_test = TensorDataset(torch.tensor(to_matrix(test, char2id, max_len=MAXLEN)))\n",
"dataloader_test = DataLoader(dataset_test, batch_size=batch, shuffle=False)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kIOckRO1GMc6"
},
"source": [
"Переопределим модель, зададим оптимизатор, лосс-функцию"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1biBu39NGNbP",
"outputId": "036d1a20-4a71-4153-e467-c1f477c868fe"
},
"outputs": [
{
"data": {
"text/plain": [
"=================================================================\n",
"Layer (type:depth-idx) Param #\n",
"=================================================================\n",
"CharGRULoop --\n",
"├─Embedding: 1-1 9,152\n",
"├─GRU: 1-2 1,036,800\n",
"├─Linear: 1-3 36,751\n",
"=================================================================\n",
"Total params: 1,082,703\n",
"Trainable params: 1,082,703\n",
"Non-trainable params: 0\n",
"================================================================="
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model = CharGRULoop(num_embeddings=len(vocab), embedding_dim=64, hidden_size=256, num_layers=3)\n",
"optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n",
"loss_fn = nn.NLLLoss(ignore_index=char2id[\"PAD\"])\n",
"# Функция потерь - negative log likelihood loss (NLL Loss) - используеся для задачи многоклассовой классификации в ситуации, когда имеются логарифмы вероятностей каждого класса\n",
"# NLL Loss эквивалентен CrossEntropyLoss, но применяется к логарифмам вероятностей - у нас на выходе линейного слоя как раз исопользуется функция softmax\n",
"\n",
"summary(model)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2nwqarvRGPY0"
},
"source": [
"Необученная модель не может делать адекватные предсказания. Ее перплексия («коэффициент неопределённости») приблизительно равна размеру словаря. Это говорит о полной неопределенности модели при генерации текста."
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "A1bX1uqvU5hn",
"outputId": "0fe2cb45-a766-4960-9994-f7276e38ce06"
},
"outputs": [
{
"data": {
"text/plain": [
"['Буря мглою небо кроет,\\nВихри снежные крутя:\\nТо, как зверь, она завоет,\\nТо заплачет, как дитя,\\nТо по кровле обветшалой\\nВдруг соломой зашумит,\\nТо, как путник запоздалый,\\nК нам в окошко застучит.\\n\\nНаша ветхая лачужка\\nИ печальна, и темна.\\nЧто же ты, моя старушка,\\nПриумолкла у окна?\\nИли бури завываньем\\nТы, мой друг, утомлена,\\nИли дремлешь под жужжаньем\\nСвоего веретена?\\n\\nВыпьем, добрая подружка\\nБедной юности моей,\\nВыпьем с горя; где же кружка?\\nСердцу будет веселей.\\nСпой мне песню, как синица\\nТихо за морем жила;\\nСпой мне песню, как девица\\nЗа водой поутру шла.\\n\\nБуря мглою небо кроет,\\nВихри снежные крутя;\\nТо, как зверь, она завоет.\\nТо заплачет, как дитя.\\nВыпьем, добрая подружка\\nБедной юности моей,\\nВыпьем с горя; где же кружка?\\nСердцу будет веселей.\\n\\n']"
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train[:1]"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1XcNzyHyGQTj",
"outputId": "14ff67ea-76c1-45ff-cb18-532d2aa0af6a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"perplexity: 141.7220001220703\n"
]
}
],
"source": [
"example = torch.tensor(to_matrix(train[:1], char2id, max_len=MAXLEN)) # Возьмем первый документ и попробуем сгененировать этот же текст начиная с первой буквы\n",
"\n",
"train_example = example[:, :-1]\n",
"target_example = example[:, 1:]\n",
"\n",
"next_logp, hid_state = model(train_example, hid_state=None)\n",
"\n",
"print('perplexity: ', torch.exp(loss_fn(next_logp.permute(0, 2, 1), target_example)).item())"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"id": "P51iPqsVGReq"
},
"outputs": [],
"source": [
"device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
"torch.cuda.empty_cache()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KiORWHsRGSiK"
},
"source": [
"Обучение. Эта ячейка займет 5-8 минут"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 522
},
"id": "naMiQGHHGTfS",
"outputId": "1c564592-0e54-4479-ccdf-260377e254dd"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABHAAAAH5CAYAAADp8cltAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACZHklEQVR4nOzdeXxcdb3/8dfMZLLvTdN0SffSvaUtFAoCBQTKjgJeAUVUuFcFl4tylftTr8hVuKJyURRBFNQroiCb7BUoBdlaSqF0o/veptn3ZJLM74/TplS6JKXNJM3r+Xh8H5OZOXPO58y3LfTd7xKKx+NxJEmSJEmS1G2FE12AJEmSJEmS9s0AR5IkSZIkqZszwJEkSZIkSermDHAkSZIkSZK6OQMcSZIkSZKkbs4AR5IkSZIkqZszwJEkSZIkSermkhJdQEe0tbWxefNmsrKyCIVCiS5HkiRJkiTpoIjH49TU1DBgwADC4b2Ps+kRAc7mzZspLi5OdBmSJEmSJEmHxIYNGxg0aNBe3+8RAU5WVhYQ3Ex2dnaCqzkwsViMZ599ltNPP51oNJrocnSI2d+9i/3du9jfvYv93bvY372L/d172Ne9S0/s7+rqaoqLi9uzj73pEQHOzmlT2dnZPTrASU9PJzs7u8f8ItKBs797F/u7d7G/exf7u3exv3sX+7v3sK97l57c3/tbMsZFjCVJkiRJkro5AxxJkiRJkqRuzgBHkiRJkiSpm+sRa+BIkiRJktQbtbW10dzcnOgyeoxYLEZSUhKNjY20trYmuhwAotEokUjkQ5/HAEeSJEmSpG6oubmZNWvW0NbWluhSeox4PE5RUREbNmzY76LAXSk3N5eioqIPVZMBjiRJkiRJ3Uw8HmfLli1EIhGKi4sJh10BpSPa2tqora0lMzOzW3xn8Xic+vp6SkpKAOjfv/8Bn8sAR5IkSZKkbqalpYX6+noGDBhAenp6osvpMXZOOUtNTe0WAQ5AWloaACUlJRQWFh7wdKrucTeSJEmSJKndzvVbkpOTE1yJDoadIVwsFjvgcxjgSJIkSZLUTXWndVx04A5GPxrgSJIkSZIkdXMGOJIkSZIkSd2cAY4kSZIkSTooZs6cyde+9rVEl3HQzJkzh1AoRGVlZaJLMcCRJEmSJEmHh7Vr15KXl8fChQsPyvmOO+44tmzZQk5OzkE534dhgCNJkiRJknqV5ubmDh2XnJxMUVFRt1hM2gBHkiRJkqRuLh6PU9/ckpAWj8cPqOaKigouv/xy8vLySE9P58wzz2TFihXt769bt45zzz2XvLw8MjIyGD9+PE8++WT7Zy+77DL69u1LWloao0aN4p577tnvNUeMGAHAtGnTCIVCzJw5E4ArrriCCy64gB/84AcMGDCA0aNHA/CHP/yBo446iqysLIqKirj00kspKSlpP98/T6G69957yc3N5ZlnnmHs2LFkZmYya9YstmzZckDfUWckHfIrSJIkSZKkD6Uh1sq47z6TkGsv+f4ZpCd3Pj644oorWLFiBY899hjZ2dl885vf5KyzzmLJkiVEo1GuvvpqmpubmTt3LhkZGSxZsoTMzEwAvvOd77BkyRKeeuopCgoKWLlyJQ0NDfu95muvvcaxxx7Ls88+y8SJE0lOTm5/77nnniM7O5vZs2e3vxaLxbjxxhsZPXo0JSUlXHvttVxxxRXtQdKe1NfX8+Mf/5g//OEPhMNhPvWpT/GNb3yDP/7xj53+jjrDAEeSJEmSJB1UO4Obf/zjHxx33HEA/PGPf6S4uJhHHnmEiy++mPXr13PhhRcyceJEAIYPH97++fXr1zNlyhSOOuooAIYOHdqh6/bt2xeAPn36UFRUtNt7GRkZ3H333buFOp/73Ofafx4+fDg/+9nPOProo6mtrW0Pk/5ZLBbjV7/6Vfton2uuuYbvf//7HarvwzDA6QLNLW186jfz2FYa4aRTW8iNRhNdkiRJkiSpB0mLRljy/TMSdu3OWrp0KUlJSRxzzDHtr/Xp04fRo0ezdOlSAL7yla/wxS9+kWeffZaPfvSjXHjhhUyaNAmAL37xi1x44YUsWLCA008/nQsuuKA9CDpQ/zwiB+DNN9/ke9/7Hm+//TYVFRW0tbUBQYA0bty4PZ4nPT29PbwB6N+//27Trg4V18DpAtFIiDfWVrCuNkRDrDXR5UiSJEmSephQKER6clJC2qFawPfKK69k9erVfPrTn2bRokUcddRR/PznPwfgzDPPZN26dfz7v/87mzdv5tRTT+Ub3/jGh7peRkbGbs/r6uo444wzyM7O5o9//CPz5s3j4YcfBva9yHH0nwZlhEKhA14nqDMMcLpAKBQiNRp81Y2xtgRXI0mSJEnSoTV27FhaWlp4/fXX218rKytj+fLlu41sKS4u5gtf+AIPPfQQX//61/n1r3/d/l7fvn35zGc+w//93//xv//7v9x11137ve7OETatrfsfPLFs2TLKysq4+eabOeGEExgzZkyXjKQ5UAY4XSQ1KRhy5ggcSZIkSdLhbtSoUZx//vlcddVVvPzyy7z99tt86lOfYuDAgZx//vkAfO1rX+OZZ55hzZo1LFiwgBdeeIGxY8cC8N3vfpdHH32UlStXsnjxYh5//PH29/alsLCQtLQ0nnnmGbZt20ZVVdVejx08eDDJycn8/Oc/Z/Xq1Tz22GPceOONB+cLOAQMcLrIzhE4TY7AkSRJkiT1Avfccw/Tpk3jnHPOYcaMGcTjcZ588sn2KUitra1cffXVjB07llmzZnHEEUfwy1/+EghG0lx//fVMmjSJE088kUgkwv3337/fayYlJXHzzTdz1113MWDAgPawaE/69u3LvffeywMPPMC4ceO4+eab+fGPf3xwbv4QcBHjLpK6Y9GnxhZH4EiSJEmSDk9z5sxp/zkvL4/f//73ez1253o3e/Ltb3+bb3/72wdUw+WXX84111xDOLxrzMq99967x2MvueQSLrnkkt1ee/96NjNnztzt+RVXXMEVV1yx2/EXXHCBa+AcTlKTgq/aKVSSJEmSJKmzDHC6SGpyMALHKVSSJEmSJB2YH/7wh2RmZu6xnXnmmYku75ByClUX2TkCp9EROJIkSZIkHZAvfOELfOITn9jje2lpaV1cTdcywOkiKdGdu1A5AkeSJEmSpAORn59Pfn7+Xt9vazt8/87tFKoukrYjwGlyEWNJkiRJktRJBjhdZNcUqsM3DZQkSZIkSYeGAU4X2TWFyhE4kiRJkiSpcwxwukhaNPiq3YVKkiRJkiR1lgFOF0ndMQKn0TVwJEmSJElSJxngdJEUtxGXJEmSJB3mZs6cyde+9rVEl3FYMsDpImnJO0bgOIVKkiRJkqRDYu3ateTl5bFw4cKDet5QKMQjjzxyUM/ZWR8qwLn55psJhUL7TNfuvfdeQqHQbi01NfXDXLZHSnUEjiRJkiRJOkAHHODMmzePO++8k0mTJu332OzsbLZs2dLe1q1bd6CX7bF27kLlCBxJkiRJUqfF49Bcl5gWjx9QyRUVFVx++eXk5eWRnp7OmWeeyYoVK9rfX7duHeeeey55eXlkZGQwfvx4nnzyyfbPXnbZZfTt25e0tDRGjRrFPffcs99rjhgxAoBp06YRCoWYOXNm+3t33303Y8eOJTU1lTFjxvDLX/6y/b3m5mauueYa+vfvT2pqKkOGDOGmm24CYOjQoQB87GMfIxQKtT/vakkH8qHa2louu+wyfv3rX/Pf//3f+z0+FApRVFR0IJc6bKS5iLEkSZIk6UDF6uGHAxJz7f/cDMkZnf7YFVdcwYoVK3jsscfIzs7mm9/8JmeddRZLliwhGo1y9dVX09zczNy5c8nIyGDJkiVkZmYC8J3vfIclS5bw1FNPUVBQwMqVK2loaNjvNV977TWOPfZYnn32WSZOnEhycjIAf/zjH/nud7/L7bffzpQpU3jrrbe46qqryMjI4DOf+Qw/+9nPeOyxx/jLX/7C4MGD2bBhAxs2bACCASyFhYXcc889zJo1i0gk0unv4mA4oADn6quv5uyzz+ajH/1ohwKc2tpahgwZQltbG1OnTuWHP/wh48eP3+vxTU1NNDU1tT+vrq4GIBaLEYvFDqTkhEsKBYllQ3Nrj70HddzOPravewf7u3exv3sX+7t3sb97F/u79+ipfR2LxYjH47S1tdHW1gZtbQlbxHbn9TsqHo+zfPlyHnvsMV566SWOO+44AP7whz8wZMgQHnroIS6++GLWr1/Pxz/+8fZ8YOfIlra2NtatW8eRRx7J1KlTARg8ePCuWvahoKAAgPz8fAoLC9s/81//9V/ccsstXHDBBQAMGTKExYsXc+edd/LpT3+adevWMWrUKI477jhCoRDFxcXtn+3Tpw8QzC56/zk7o62tjXg8TiwW+0AA1NFfm50OcO6//34WLFjAvHnzOnT86NGj+e1vf8ukSZOoqqrixz/+MccddxyLFy9m0KBBe/zMTTfdxA033PCB15999lnS09M7W3K3sKwyBETYXlHVPiRMh7/Zs2cnugR1Ifu7d7G/exf7u3exv3sX+7v36Gl9nZSURFFREbW1tTQ3NwfTmK5emphiGlqgsbpDh7a0tNDc3Mybb75JUlISY8eObR+UEY1GGTlyJG+//TZnnHEGV155JV//+td56qmnmDlzJueeey4TJkwA4PLLL+czn/kM8+fP5+STT+bss8/mmGOO2e/16+rqAKivr2+/bl1dHatWreKqq67i3/7t33arNTs7m+rqai666CI+9rGPMXr0aE499VTOOOMMTjnllN2/hoaG9nN2VnNzMw0NDcydO5eWlpbd3quvr+/QOToV4GzYsIGvfvWrzJ49u8MLEc+YMYMZM2a0Pz/uuOMYO3Ysd955JzfeeOMeP3P99ddz7bXXtj+vrq6muLiY008/nezs7M6U3G30WbWdO5a+RTQ1nbPOOiHR5egQi8VizJ49m9NOO41oNJrocnSI2d+9i/3du9jfvYv93bvY371HT+3rxsZGNmzYQGZm5vv+/p2T0Jo6IikpieTk5PbBF9nZ2buNOIlEIqSkpJCdnc0111zD+eefzxNPPMHs2bM55ZRT+PGPf8w111zDhRdeyIknnsiTTz7J3//+dy644AK+9KUvccstt+zz+hkZwVSv9PT09vxg59SrO++88wMhUCQSITs7mxNOOIHVq1fz1FNP8dxzz/G5z32OU089lQceeKD92LS0tAPOJBobG0lLS+PEE0/8QJ7S0VCoUwHOm2++SUlJSfsQJoDW1lbmzp3L7bffTlNT037ngkWjUaZMmcLKlSv3ekxKSgopKSl7/GxP+g33fplpwf00tbT12HtQ5/XkX7PqPPu7d7G/exf7u3exv3sX+7v36Gl93draSigUIhwOEw4navLUgQmFQowfP56WlhbmzZvXPoWqrKyM5cuXM378+PZ7GjJkCF/60pf40pe+xPXXX8/dd9/NV77yFQD69evHZz/7WT772c9y5513ct111/GTn/xkn9femSW0tbW1X6N///4MGDCAtWvX8ulPf3qvn83NzeWSSy7hkksu4eKLL2bWrFlUVlaSn59PNBolHo8fcF+Ew2FCodAefx129NdlpwKcU089lUWLFu322mc/+1nGjBnDN7/5zQ4t5NPa2sqiRYs466yzOnPpHi/FbcQlSZIkSb3EqFGjOP/887nqqqu48847ycrK4lvf+hYDBw7k/PPPB+BrX/saZ555JkcccQQVFRW88MILjB07FoDvfve7TJs2jfHjx9PU1MTjjz/e/t6+FBYWkpaWxjPPPMPgwYNJTU0lJyeHG264ga985Svk5OQwa9YsmpqamD9/PhUVFVx77bX89Kc/pX///kyZMoVwOMwDDzxAUVERubm5QLA+z3PPPcfxxx9PSkoKeXl5h+y725tORUdZWVlMmDBht5aRkUGfPn12m6d2/fXXt3/m+9//Ps8++yyrV69mwYIFfOpTn2LdunVceeWVB/dOurm0ZLcRlyRJkiT1Hvfccw/Tpk3jnHPOYcaMGcTjcZ588sn2ESetra1cffXVjB07llmzZnHEEUe0b+2dnJzM9ddfz6RJkzjxxBOJRCLcf//9+71mUlISN998M3fddRcDBgxoD4uuvPJK7r77bu655x4mTpzISSedxL333suwYcOAIO/40Y9+xFFHHcXRRx/N2rVrefLJJ9tH3PzkJz9h9uzZFBcXM2XKlEPxde3/3g72CdevX7/bkKKKigquuuoqtm7dSl5eHtOmTeOVV15h3LhxB/vS3VrqjhE4TS1ttLXFCYdDCa5IkiRJkqSDa86cOe0/5+Xl8fvf/36vx/785z/f63vf/va3+fa3v31ANVx++eVcc801H5judOmll3LppZfu8TNXXXUVV1111V7Pee6553LuueceUD0Hy4cOcN7fOXt6fuutt3Lrrbd+2Mv0eCnRXdPLmlra2kfkSJIkSZIk7U/PWgmpB9s5AgdcB0eSJEmSpAPxwx/+kMzMzD22M888M9HlHVIHfQqV9iwpEiYSitMaD9HYYoAjSZIkSVJnfeELX+ATn/jEHt9LS0vr4mq6lgFOF4qGobUVGpoNcCRJkiRJ6qz8/Hzy8/P3+n5b2+G7cZBTqLpQ8o5v252oJEmSJEkdEY/HE12CDoKD0Y8GOF0oujPAcQqVJEmSJGkfIpFg45vm5uYEV6KDob6+HqB9C/UD4RSqLtQe4DiFSpIkSZK0D0lJSaSnp7N9+3ai0egHtsTWnrW1tdHc3ExjY2O3+M7i8Tj19fWUlJSQm5vbHswdCAOcLuQIHEmSJElSR4RCIfr378+aNWtYt25dosvpMeLxOA0NDaSlpREKhRJdTrvc3FyKioo+1DkMcLqQa+BIkiRJkjoqOTmZUaNGOY2qE2KxGHPnzuXEE0/8UNOVDqZoNPqhRt7sZIDThaLhOBByFypJkiRJUoeEw2FSU1MTXUaPEYlEaGlpITU1tdsEOAdL4ieE9SJOoZIkSZIkSQfCAKcLRZ1CJUmSJEmSDoABThdK3jHlrTHmCBxJkiRJktRxBjhdaNcIHAMcSZIkSZLUcQY4XcgAR5IkSZIkHQgDnC60cxvxBgMcSZIkSZLUCQY4XSjYRtxFjCVJkiRJUucY4HQhp1BJkiRJkqQDYYDThZINcCRJkiRJ0gEwwOlCu0bgOIVKkiRJkiR1nAFOF3IKlSRJkiRJOhAGOF3IXagkSZIkSdKBMMDpQrt2oTLAkSRJkiRJHWeA04VcA0eSJEmSJB0IA5wuFI0Ej47AkSRJkiRJnWGA04XcRlySJEmSJB0IA5wu1D6FqqWNeDye2GIkSZIkSVKPYYDThXYGOK1tcWKtBjiSJEmSJKljDHC6UPL7vu3GFqdRSZIkSZKkjjHA6UKREIRCwc+ugyNJkiRJkjrKAKcLhUKQtmMrqsZmtxKXJEmSJEkdY4DTxVKSgq/cKVSSJEmSJKmjDHC6WOrOEThOoZIkSZIkSR1kgNPF0nZsRdXQbIAjSZIkSZI6xgCni6Uk7RiB0+IaOJIkSZIkqWMMcLpY6o4ROE6hkiRJkiRJHWWA08XSXANHkiRJkiR1kgFOF0txBI4kSZIkSeokA5wulrpjDRwXMZYkSZIkSR1lgNPFUpNdxFiSJEmSJHWOAU4XS01yCpUkSZIkSeocA5wulrpjEeMGAxxJkiRJktRBBjhdbOc24k0xp1BJkiRJkqSOMcDpYjsXMXYKlSRJkiRJ6igDnC62cwSOU6gkSZIkSVJHGeB0sZ1r4DgCR5IkSZIkdZQBThfbOQKn0TVwJEmSJElSBxngdLGda+A4hUqSJEmSJHWUAU4XS00OApwmAxxJkiRJktRBBjhdLDXJKVSSJEmSJKlzDHC62M5FjJ1CJUmSJEmSOsoAp4vtWsTYAEeSJEmSJHWMAU4X27mIsQGOJEmSJEnqKAOcLuY24pIkSZIkqbMMcLrYzjVwmlvbaG2LJ7gaSZIkSZLUExjgdLGdI3AAmlqcRiVJkiRJkvbPAKeL7VwDB6Ch2QBHkiRJkiTt34cKcG6++WZCoRBf+9rX9nncAw88wJgxY0hNTWXixIk8+eSTH+ayPVo4HCI5acc6OC2ugyNJkiRJkvbvgAOcefPmceeddzJp0qR9HvfKK69wySWX8PnPf5633nqLCy64gAsuuIB33333QC/d46UmuZW4JEmSJEnquAMKcGpra7nsssv49a9/TV5e3j6Pve2225g1axbXXXcdY8eO5cYbb2Tq1KncfvvtB1Tw4WDnQsZOoZIkSZIkSR2RdCAfuvrqqzn77LP56Ec/yn//93/v89hXX32Va6+9drfXzjjjDB555JG9fqapqYmmpqb259XV1QDEYjFisdiBlJxwO+uOxWLtCxnXNTb32PvRvr2/v3X4s797F/u7d7G/exf7u3exv3sP+7p36Yn93dFaOx3g3H///SxYsIB58+Z16PitW7fSr1+/3V7r168fW7du3etnbrrpJm644YYPvP7ss8+Snp7euYK7mdmzZxNriAAhXvzHq2x9163ED2ezZ89OdAnqQvZ372J/9y72d+9if/cu9nfvYV/3Lj2pv+vr6zt0XKcCnA0bNvDVr36V2bNnk5qaekCFdcT111+/26id6upqiouLOf3008nOzj5k1z2UYrEYs2fP5rTTTuM3G95ky8ZqJk2ZxqljChNdmg6B9/d3NBpNdDk6xOzv3sX+7l3s797F/u5d7O/ew77uXXpif++cdbQ/nQpw3nzzTUpKSpg6dWr7a62trcydO5fbb7+dpqYmIpHIbp8pKipi27Ztu722bds2ioqK9nqdlJQUUlJSPvB6NBrtMR2wN9FolLRo8LW3xEM9/n60b4fDr1l1nP3du9jfvYv93bvY372L/d172Ne9S0/q747W2alFjE899VQWLVrEwoUL29tRRx3FZZddxsKFCz8Q3gDMmDGD5557brfXZs+ezYwZMzpz6cPKzkWMG2NuIy5JkiRJkvavUyNwsrKymDBhwm6vZWRk0KdPn/bXL7/8cgYOHMhNN90EwFe/+lVOOukkfvKTn3D22Wdz//33M3/+fO66666DdAs9T9rOXajcRlySJEmSJHXAAW0jvi/r169ny5Yt7c+PO+447rvvPu666y4mT57Mgw8+yCOPPPKBIKg32bkLVZMBjiRJkiRJ6oAD2kb8/ebMmbPP5wAXX3wxF1988Ye91GFj1xQqAxxJkiRJkrR/B30EjvYv1SlUkiRJkiSpEwxwEsBFjCVJkiRJUmcY4CTAzjVwnEIlSZIkSZI6wgAnAdyFSpIkSZIkdYYBTgLsnELV5BQqSZIkSZLUAQY4CeAUKkmSJEmS1BkGOAngLlSSJEmSJKkzDHASYNcuVAY4kiRJkiRp/wxwEsBtxCVJkiRJUmcY4CRAmiNwJEmSJElSJxjgJICLGEuSJEmSpM4wwEmA9ilULU6hkiRJkiRJ+2eAkwA7p1A1NDsCR5IkSZIk7Z8BTgKk7JxC1dJKPB5PcDWSJEmSJKm7M8BJgJ1TqOJxaG51GpUkSZIkSdo3A5wE2DmFCqCx2QBHkiRJkiTtmwFOAkQjYSLhEBBMo5IkSZIkSdoXA5yu0FRD+IUfMHXtr4J5U0BqkluJS5IkSZKkjjHA6QqRFMKv3kZxxStQuw2AtOQdO1EZ4EiSJEmSpP0wwOkKScmQMxiAUMVqAFKSggCnMeYaOJIkSZIkad8McLpIPH948EP5GgBSo06hkiRJkiRJHWOA00XiecOAXSNwnEIlSZIkSZI6ygCnq+wYgRMqDwKc1B1TqJoMcCRJkiRJ0n4Y4HSR+D8HOFHXwJEkSZIkSR1jgNNFdk6homINxOPtAY5TqCRJkiRJ0v4Y4HSV3CG0ESYUq4earS5iLEmSJEmSOswAp6tEotQnFwQ/l69yCpUkSZIkSeowA5wuVJdSFPxQtoo0p1BJkiRJkqQOMsDpQnUphcEP5avbp1C5C5UkSZIkSdofA5wu1D4CZ7cpVAY4kiRJkiRp3wxwulBtSr/gh7LV7kIlSZIkSZI6zACnC9XtDHDKV5OatHMXKhcxliRJkiRJ+2aA04XqUwqIhyLQ0kBeWyngFCpJkiRJkrR/BjhdKB5KgtzBABQ0bQScQiVJkiRJkvbPAKeLxfNHAJDXuAGAJqdQSZIkSZKk/TDA6WLxvGEAZNcHAU5jiyNwJEmSJEnSvhngdLX84QBk1q0DoKHZAEeSJEmSJO2bAU4Xi+8IcNJrgwDHETiSJEmSJGl/DHC62M4pVCnVawnR5jbikiRJkiRpvwxwulruYAgnEW5toh8VNDqFSpIkSZIk7YcBTlcLJ0HuEACGhbc6hUqSJEmSJO2XAU4i9Am2Eh8a2kqsNU5Lq9OoJEmSJEnS3hngJMKOhYxHJZUA8PbGqkRWI0mSJEmSujkDnETID0bgHJ1dAcBfF2xMZDWSJEmSJKmbM8BJhD7BCJzh4W0APP72ZhpjroUjSZIkSZL2zAAnEXZMoUqv28DA7GSqG1t4bmlJgouSJEmSJEndlQFOIuQEW4mHWhr59IQo4DQqSZIkSZK0dwY4iRBJgryhAJxX3AjAi+9tZ3tNUwKLkiRJkiRJ3ZUBTqLsmEY1oHUzRxbn0toW59GFmxJclCRJkiRJ6o4McBJlx05UlK3iwqkDAXhogQGOJEmSJEn6IAOcROmzI8ApX825kweQHAmzZEs1S7dUJ7YuSZIkSZLU7RjgJMqOKVSUryY3PZlTxxYC8Nc3XcxYkiRJkiTtzgAnUd4X4FCxlo9PHQTAIws309LalsDCJEmSJElSd2OAkyi5g6FwHLQ2w2/PZGZ+OfkZyZTWNvHSitJEVydJkiRJkroRA5xECUfgUw9B37FQs5no787i30YF69/8dYHTqCRJkiRJ0i4GOImU3R8++yQMmAoN5Vy56itMDy3lyUVb+NvbmxNdnSRJkiRJ6iY6FeDccccdTJo0iezsbLKzs5kxYwZPPfXUXo+/9957CYVCu7XU1NQPXfRhJT0fPvMYDD2BSKyWP6b+D+eFXuabf36DpxZtSXR1kiRJkiSpG+hUgDNo0CBuvvlm3nzzTebPn88pp5zC+eefz+LFi/f6mezsbLZs2dLe1q1b96GLPuykZMFlD8ARZxKNN/O/yb/kreiVZD7wCZY/fBOULIN4PNFVSpIkSZKkBEnqzMHnnnvubs9/8IMfcMcdd/Daa68xfvz4PX4mFApRVFR04BX2FtE0+Jc/wPP/TXzRg6RUb+SE0Dvw9jvw9s3Bosdjz4NxF8DAaRB29pskSZIkSb1FpwKc92ttbeWBBx6grq6OGTNm7PW42tpahgwZQltbG1OnTuWHP/zhXsOenZqammhqamp/Xl0dLO4bi8WIxWIHWnJC7ax7v/XP/Dac9P9oLVnG44/cR+G2lzkmvIyUyvXw6u3w6u3Es/rTNuZc4sNPJt5vImQZkHU3He5vHRbs797F/u5d7O/exf7uXezv3sO+7l16Yn93tNZQPN65uTmLFi1ixowZNDY2kpmZyX333cdZZ521x2NfffVVVqxYwaRJk6iqquLHP/4xc+fOZfHixQwaNGiv1/je977HDTfc8IHX77vvPtLT0ztTbo/WGoffvxdmWXmMk8Jvc1bkDU6NLCCTxt2Oa0zKoSp9CJVpQ6lJHUB9SiF1yYU0J2VBKJSg6iVJkiRJ0v7U19dz6aWXUlVVRXZ29l6P63SA09zczPr166mqquLBBx/k7rvv5sUXX2TcuHH7/WwsFmPs2LFccskl3HjjjXs9bk8jcIqLiyktLd3nzXRnsViM2bNnc9pppxGNRjv+udY2bntuFX9fVsKq7XWk0MxHwos4IzyfKeGVDA9tJhLacxc2hVIpSepPc3o/8voOIKdPf8jsSzy9L+QPI14wBlJ75vfZ3R1of6tnsr97F/u7d7G/exf7u3exv3sP+7p36Yn9XV1dTUFBwX4DnE5PoUpOTmbkyJEATJs2jXnz5nHbbbdx55137vez0WiUKVOmsHLlyn0el5KSQkpKyh4/31M6YG86ew/RKFx/9jiuP3sclfXNvLW+kvnrxvLQutP4wZYamhtqGBtaz/jwWsaH1jI0vI3iUAn9KSeFRopja6BqDVQBe/raswdC4digDZsJw0+CSM/+jruTw+HXrDrO/u5d7O/exf7uXezv3sX+7j3s696lJ/V3R+s84DVwdmpra9tttMy+tLa2smjRor1OudK+5aYnc/KYQk4eUwhAPB5ne00T722rZfm2Gt4uqeGtNshKTSInuY2ieCn5zZvYuH4t27ZuJC9eSZ9QNYVUMDK8laJQGVRvCtrKv8MrP6cuksOi7BNZkH0Kq9OP5OwjB3Hy6MIE37kkSZIkSb1bpwKc66+/njPPPJPBgwdTU1PDfffdx5w5c3jmmWcAuPzyyxk4cCA33XQTAN///vc59thjGTlyJJWVldxyyy2sW7eOK6+88uDfSS8UCoUozE6lMDuVj4wq2OexNY0xnl28jUff3szLK0tpjcXJpo6RoU2MDm9gYmgNp0Xm07e1imMr/saxFX+jJJ7LPe/M4okJn+fb5x9JbnpyF92ZJEmSJEl6v04FOCUlJVx++eVs2bKFnJwcJk2axDPPPMNpp50GwPr16wm/b3vriooKrrrqKrZu3UpeXh7Tpk3jlVde6dB6OTq4slKjXDhtEBdOG0RdUwvbqhspq2umtKaJ0rpmttc288e2GEOrF3BE2WyGb3+ewpZKvhm9n9VL5/DdFZ/nrI9/mlkT+if6ViRJkiRJ6nU6FeD85je/2ef7c+bM2e35rbfeyq233trponRoZaQkMbxvJsP77undccCnoKUZ3v0rsWe+w/CGrfys7QfM/vNT/Ne8a/nyxadTkPnBNYokSZIkSdKhEd7/IeqVkpLhyEuIfnUBLcdeTWsowmmRBfzn2s9yx0++zaMLN9HJDcwkSZIkSdIBMsDRvqVmkzTrh0S+9Cq1Az9CSijGd+J3sfCBm7jq9/PZWtWY6AolSZIkSTrsGeCoY/qOJvPKx2k97msA/Ff0D4x47zecduuL/GX+BkfjSJIkSZJ0CH3obcTVi4RCRE77HkRT4cWbuT76J6KxFv7jwY/x7qYqbjhvPKFQKNFVSpIkSZJ02DHAUeeEQnDy9RBJguf/m29EHyA51MJPX72IwfnpXHnC8ERXKEmSJEnSYccARwfmxOsgkgKzv8NXkh6mMR7lB0+GKM5P54zxRYmuTpIkSZKkw4pr4OjAHf8VOOMmAL4RfYCTQgv52v0LeWdjZWLrkiRJkiTpMGOAow9nxpfgqM8TJs4vUn5J35bNfP5389lU2ZDoyiRJkiRJOmwY4OjDm3UzDDqajHgtv0u7jdqaKj53zzxqGmOJrkySJEmSpMOCAY4+vKRk+MQfIKOQYW1r+d+037J8WzVfvX8hrW1uLy5JkiRJ0odlgKODI7s/fOJ3EE7ijPjL/Gv0GZ5fVsJPZy9PdGWSJEmSJPV4Bjg6eIYcB2f8EIBvJf2Ro0PL+MULq3jinS0JLkySJEmSpJ7NAEcH1/R/hUn/Qjjeyq9y7iWZGN944G2WbqlOdGWSJEmSJPVYBjg6uEIhOOsWyOxHn8b1/LDweRpirfzrH+ZTUdec6OokSZIkSeqRDHB08KXmtE+lurD+zxyTW82G8gauvm8BLa1tCS5OkiRJkqSexwBHh8aEC2HYiYRaGvlN4V9ITw7zyqoy7n55TaIrkyRJkiSpxzHA0aERCsFZP4FwlMz1z3PX0VsB+O3La2hucRSOJEmSJEmdYYCjQ6fvEXD8VwE4fsUtDM1qo6Smicff2ZzgwiRJkiRJ6lkMcHRonfB1yB1MqHoTPy2aDcCvX1pDPB5PcGGSJEmSJPUcBjg6tJLT4cxbAJiy6Y9MjG5m6ZZqXl1VluDCJEmSJEnqOQxwdOiNngWjzybU1sKP8h8D4NcvrU5wUZIkSZIk9RwGOOoaH/0vIMTYqrmMC6/jheXbWVlSk+iqJEmSJEnqEQxw1DX6joYJHwfgxrwnAfjNy2sTWJAkSZIkST2HAY66zonXASGm1b3E6NB6HlqwkbLapkRXJUmSJElSt2eAo65TOBbGnQ/Ad7KeoKmljf97bX2Ci5IkSZIkqfszwFHXOvE6AI5vfpmRoY384bW1NMZaE1yUJEmSJEndmwGOulbRBBhzDiHi/Efa3yitbebhtzYluipJkiRJkro1Axx1vZO+CcBpbS8zIrSJX7ywkuaWtgQXJUmSJElS92WAo67XfxKMPosQcb6e+jc2VjTw1wUbE12VJEmSJEndlgGOEuOk/wBgVvxlhoa2cPvzjsKRJEmSJGlvDHCUGAOmwKgzCNPGdamPsamygQffdBSOJEmSJEl7YoCjxDn5egDOis9lZGija+FIkiRJkrQXBjhKnAFT2nek+lbqw2yqbOCBNzckuipJkiRJkrodAxwl1sn/CYT4aPxVxoXW8ovnV9LU0proqiRJkiRJ6lYMcJRY/cbDhI8D8K3Uh9lc1chf5rsWjiRJkiRJ72eAo8SbeT2EwpwYn8fk0Ep++YKjcCRJkiRJej8DHCVewSiYfAkA16c+xJaqRv70+voEFyVJkiRJUvdhgKPu4aT/gHASx8YXMj20lJ8/v5LappZEVyVJkiRJUrdggKPuIW8oTPk0AP8v7SHK6pr49dzVia1JkiRJkqRuwgBH3ceJ10Ekhcltizkx/A6/fmk122uaEl2VJEmSJEkJZ4Cj7iNnIBx9JQA/SLuP5uYmfvbcigQXJUmSJElS4hngqHs56T8gvYDi1g18JvIMf3pjPWtK6xJdlSRJkiRJCWWAo+4lLRc++l8AfCP5YfLaKvjxs8sTW5MkSZIkSQlmgKPu58hPwYCppMXr+Vb0Tzzxzhbe3lCZ6KokSZIkSUoYAxx1P+EwnHULABdGXmJq6D1ufmoZ8Xg8wYVJkiRJkpQYBjjqngYdBVM+BcD3k3/H66u388LykgQXJUmSJElSYhjgqPs69XuQksOE0Br+JfICN/xtCY2x1kRXJUmSJElSlzPAUfeV2RdOvh6Ab0X/TFXZNu58cXWCi5IkSZIkqesZ4Kh7O/pK6DuWHGq5JXoXd8x5j/Vl9YmuSpIkSZKkLmWAo+4tEoWP3UE8ksxpkTe5Iv4Y3/vbYhc0liRJkiT1KgY46v4GTCF05o8AuC7pzzS89wKzl2xLcFGSJEmSJHUdAxz1DNOugMmXEAnF+Vn05/zisZdpaHZBY0mSJElS72CAo54hFIKzf0pb33H0DVXz/xp+xB3PLUt0VZIkSZIkdQkDHPUcyemEP/l/xJIymR5eTu4rP2D51ppEVyVJkiRJ0iFngKOepc8Ikj5+BwCfizzBg7+9hfK65gQXJUmSJEnSoWWAox4nNO48Go75CgD/0XQ7d979K5pb2hJclSRJkiRJh44BjnqktDNuoHrUx4iGWvlq+X9z9/1/cWtxSZIkSdJhq1MBzh133MGkSZPIzs4mOzubGTNm8NRTT+3zMw888ABjxowhNTWViRMn8uSTT36ogiUAwmGy/+UuyopOID3UxCdXfJ1H/v5ioquSJEmSJOmQ6FSAM2jQIG6++WbefPNN5s+fzymnnML555/P4sWL93j8K6+8wiWXXMLnP/953nrrLS644AIuuOAC3n333YNSvHq5pGT6fPZ+SrLGkx+q5eiXP88bb+/516IkSZIkST1ZUmcOPvfcc3d7/oMf/IA77riD1157jfHjx3/g+Ntuu41Zs2Zx3XXXAXDjjTcye/Zsbr/9dn71q1/t9TpNTU00NTW1P6+urgYgFosRi8U6U3K3sbPunlp/txVOIfdzD1Lyy9MYFNtI/cOfZFnm3xgxeGBCy7K/exf7u3exv3sX+7t3sb97F/u797Cve5ee2N8drTUUP8CFQ1pbW3nggQf4zGc+w1tvvcW4ceM+cMzgwYO59tpr+drXvtb+2n/913/xyCOP8Pbbb+/13N/73ve44YYbPvD6fffdR3p6+oGUq8NctGE7xyy7kT5U8m58OAuOuI6czIxElyVJkiRJ0j7V19dz6aWXUlVVRXZ29l6P69QIHIBFixYxY8YMGhsbyczM5OGHH95jeAOwdetW+vXrt9tr/fr1Y+vWrfu8xvXXX8+1117b/ry6upri4mJOP/30fd5MdxaLxZg9ezannXYa0Wg00eUclmrWT6Xy/85nAqsJrfwpmZ9/hAH/9Ouvq9jfvYv93bvY372L/d272N+9i/3de9jXvUtP7O+ds472p9MBzujRo1m4cCFVVVU8+OCDfOYzn+HFF1/ca4hzIFJSUkhJSfnA69FotMd0wN4cDvfQXeWPmEr5px+l6vfnMT6+gnd/+3Eqv/QkfQsKElaT/d272N+9i/3du9jfvYv93bvY372Hfd279KT+7midnd5GPDk5mZEjRzJt2jRuuukmJk+ezG233bbHY4uKiti2bdtur23bto2ioqLOXlbqkPzhU2m+7CGqyWRC23K23nEu5RXliS5LkiRJkqQPpdMBzj9ra2vbbcHh95sxYwbPPffcbq/Nnj2bGTNmfNjLSnvVd9R06v7lQWpIZ2LrEjb94lzWbNqS6LIkSZIkSTpgnQpwrr/+eubOncvatWtZtGgR119/PXPmzOGyyy4D4PLLL+f6669vP/6rX/0qTz/9ND/5yU9YtmwZ3/ve95g/fz7XXHPNwb0L6Z/0HzuDygv/Qi3pTGx5l8a7TucXj7xIXVNLokuTJEmSJKnTOhXglJSUcPnllzN69GhOPfVU5s2bxzPPPMNpp50GwPr169myZddIh+OOO4777ruPu+66i8mTJ/Pggw/yyCOPMGHChIN7F9IeFE88gZp/eYiqcB5jQ+u58K3P8IVb7uXRhZs4wM3XJEmSJElKiE4tYvyb3/xmn+/PmTPnA69dfPHFXHzxxZ0qSjpY+o+dQfzLL1J7z4UUVa/gjtj/45q/bOP+N07ll5dNJS8jOdElSpIkSZK0Xx96DRypuwvlDSHzi3+ndehJZIYauTv6Y4av+zOX3v065XXNiS5PkiRJkqT9MsBR75CWS+RTD8KRnyIp1MYPor/l/O2/4rK7XjHEkSRJkiR1ewY46j2SkuH82+HkbwPwhaTHuab8B1xx14uGOJIkSZKkbs0AR71LKAQnXQcf/zXxcDJnR97gexXf4ot3PkNZbVOiq5MkSZIkaY8McNQ7TfoEocsfpjUll6nhlfyo8lqu+9WDrCurS3RlkiRJkiR9gAGOeq+hHyFy1d+JZQ9hSLiEW2u+we0/+x+efndLoiuTJEmSJGk3Bjjq3QpGEf2352nufzQ5oXpuCd1Gy5+v4JaHX6W5pS3R1UmSJEmSBBjgSJBRQPKVT9F64rdoI8I5kdf4zMJP8j8//xmbKhsSXZ0kSZIkSQY4EgCRKJFTrid81d+pzRpOYaiS71T9F6//72W89M7KRFcnSZIkSerlDHCk9xs4lcyvvELNkf8KwMd5jtF/PZVH//gLWlpaE1ycJEmSJKm3MsCR/lk0jawLbqH5039je8pgCkOVnL/iP3n7llls37gi0dVJkiRJknohAxxpL5JHnEjf6+azfMzVxOIRpjW9Qebdx7Pmsf+B1pZElydJkiRJ6kUMcKR9SUph9Cd/yNZLn2NRZDxpNDFswQ8pufV44pveSnR1kiRJkqRewgBH6oDi0VMY9c0XeWDAf1AVT6ewdhnxX59C0+P/AU21iS5PkiRJknSYM8CROig1OcpFV/0nz3/0Cf7Wdhxh2kiZfyexn0+H5U8nujxJkiRJ0mHMAEfqhFAoxMdOmErxVX/i36PfZkNbX6K1m+BP/wJ/uRxqtia6REmSJEnSYcgARzoARxbn8u2vfoX/GnQ3v2o5h5Z4GJY8Svz2o2De3RBvS3SJkiRJkqTDiAGOdID6ZKZw1+dPpPaE73J+7L9Z2DacUFMNPPF1Ir87m6yGDYkuUZIkSZJ0mEhKdAFST5YUCfONM0Yzc3RfvvrnI5hZ/TeuS/ozmZvmcTLziT++BE79NmQPSHSpkiRJkqQezBE40kFw1NB8Hv/qTBqmfJ6PNt3Ck63TCREn/PYf4WdT4e83QGNVosuUJEmSJPVQBjjSQZKVGuVHF03me586nf8XvY6PN32PeW1HQEsDvPxTuO1IeO0OaGlOdKmSJEmSpB7GAEc6yGZNKOKZrx5PRr+RfLLle1zVfC2r4gOgoRye/hbcfhQsehDaXOhYkiRJktQxBjjSIZCXnsxFw9r429XH0TTyTE5v+h+uj32e7eRC5Tr46+fh1yfD6hcTXaokSZIkqQcwwJEOoVGFmfz+c9O5+7PH8kb+eZzY+FN+HLuYOlJhy0L4/Xnw+wtgwxuJLlWSJEmS1I25C5XUBU4eXciJo/ry2NubuO3v+fyp7BS+nPQwn0p6jqTVL8DqF2DEKXDSt2DwMYkuV5IkSZLUzRjgSF0kEg7xsSmDOHfSAB5ZuJmfPdef31ScydWRR7ko6SWSVj0Pq56H4SfDzG/B4GMTXbIkSZIkqZtwCpXUxZIiYS6aNojnvn4SX7jgVG6KXs3Mpp/wp5aTaSUSjMb57Rnw+/Nh3auJLleSJEmS1A0Y4EgJEo2EueyYIbzwjZkcP20a17dcxUlNP+VBTt0R5MyBe2ZRf/fZNK3+R6LLlSRJkiQlkAGOlGD5Gcn8z0WT+OsXZ5BVNIJvNH6ek5p+yn0tJxOLR0jf+DIpvz+LdTcfS8srd0DNtkSXLEmSJEnqYq6BI3UT04bk87drjufxd7aweHMVL5ZP4Jntazm76j4+xhyGNC6FZ79FfPZ/Ehp2Eky8GMadBylZiS5dkiRJknSIGeBI3UhSJMwFUwZywZSBO145inj8Qv7x9lJe+OudnB16manhlcE6OatfgCevg4kXwVGfhQFTElq7JEmSJOnQMcCRurlQKMRHjhwHGd/h0t/Po2/TFr5S+DYXJv2DcNkKWPC7oA2YAtM+G4zMSU5PdNmSJEmSpIPINXCkHuIjowq454rplCYN4Lptp/Pp1J/TcNljMOEiiCTD5rfgb1+B24+GxQ9DPJ7okiVJkiRJB4kBjtSDzBjRh999bjoZyRH+saqcCffUcfLaT/O1gX/i+cFfpi6tP1RvhAeugN+fByVLE12yJEmSJOkgMMCRepjpw/L5/een0z8nlda2OGtK63jkvSY+994MplbcxF2hi2mNpMCauXDH8fDUt6ChItFlS5IkSZI+BNfAkXqgaUPyeeVbp1BS08TKktr29o+Vpfyw9GP8PnQ8d/b9K+OrX4LX74C3/hAsdHzs1ZDdP9HlS5IkSZI6yQBH6qFCoRD9slPpl53K8SMLAGiMtfLjZ5Zz98twdskXuSjnFG5M/zNpFcvglZ/D63fC5E/CcV+FgpEJvgNJkiRJUkc5hUo6jKRGI3z7nHHcd+Ux9M9J5cGq0UzY9l3uGfIj6oqmQ2szLPg93H4U/OXyYOFjSZIkSVK3Z4AjHYaOG1nA0189kfMmD6C1DW5YPojxa7/GN3NuYVPhTCAOSx6Fu2bC78+H1XPctUqSJEmSujEDHOkwlZMe5WeXTOEv/zaD8yYPIBoJ8edtAzl+/b/yMX7M4oIziYciQXjz+/Ph1yfD0r9BW1uiS5ckSZIk/RPXwJEOc9OH5TN9WD6lteN48M2N3Pf6et4qH8DZGz/N6NRz+NGAuUwqeYzQ5rfgz5+CvmPhhK/D+I9BxD8iJEmSJKk7cASO1EsUZKbwhZNGMOcbM7nz09MY3S+L5Y15nL/6fD7adjsLh15JPCULti+Fh66EXxwNC/4ALc2JLl2SJEmSej0DHKmXCYdDnDG+iCe/egK3ffJIhvZJZ1V9GhcsO4VzI3ewZtK1kJYP5avhsWvgZ1PgjV9DrCHRpUuSJElSr2WAI/VSkXCI848cyOxrT+Lmj0+kMCuFd8vDnPzGUXyt/x+oOvF7kNkPqjfCk9+A2ybDP34GTbWJLl2SJEmSeh0DHKmXi0bCfHL6YJ77+kl8/iPDiIRDPLKkihlzxvLrqY9QefJNxHMGQe02mP0duHU8/OUz8MrtsP51iDUm+hYkSZIk6bDnCqWSAMhKjfKdc8Zx0bRBfOeRd5m/roIfPLuGHzCEvNQfcWX2PP6l8QEKGjfCkkeCBhCOwoAj4SPXwpizEngHkiRJknT4cgSOpN2M7Z/NX/5tBrdcNIkxRVkkhUNUNMItJUdzTPVN/EvTd/ht6uXUDz8DMvpCWww2zoP7L4E/XQqVGxJ9C5IkSZJ02HEEjqQPCIdDXHxUMRcfVUxzSxurttfy3rYalm2t4cE30/l+5Vh+3hzll5dOZUafOpj/W3j1dlj+BKx+AWZ+C479EkSiib4VSZIkSTosOAJH0j4lJ4UZ2z+b848cyDdnjeGxa45n4sAcKupjfPq3b/B/y4HTboAvvAyDj4NYPcz+Ltx5Iix5FNpaE30LkiRJktTjGeBI6pT+OWk88IUZnDd5AC1tcb79yLt8+5FF1OeOgs8+Cef/MtiGvGQJ/OVyuP0omPcbtyGXJEmSpA/BAEdSp6VGI9z2ySP5j1mjCYXg/15bz4ybnueHTy1jw5CPwZffhBOvg9RcKF8NT1wLt06AOTdD2apEly9JkiRJPY4BjqQDEgqF+NLMkfzmM0cxOD+dqoYYd81dzUm3vMC/PriaV4Z8gfi/L4YzfwQ5g6G+FObcBD+fCrdPh7/fABvfhLa2RN+KJEmSJHV7LmIs6UM5ZUw/TjqikDnLS7j3lbW8tKKUZ5ds49kl25g+LJ//POtfOPIrnw+2HX/r/2DtS1C6HF5eDi//FLIHwnFfgWlXQDQ10bcjSZIkSd2SAY6kDy0SDnHq2H6cOrYfK0tq+N0r6/jL/A28saacC37xD86e1J//OONMhky8CBoqYeXfYdnjsGI2VG+Cp78J/7gNTrgWpl4OSSmJviVJkiRJ6lacQiXpoBpZmMWNF0zghW/M5KJpgwiF4Il3tnDqT17ke48tpqw1DSZeBBffC9etgnP+F7IHQc1mePIb8PNpwbbkTbWJvhVJkiRJ6jYMcCQdEgNy0/jxxZN58isnMHN0X1ra4tz7ylpOumUOtz+/gvrmlmDK1FGfha8sgLN+DJlFULUBHv93+MloeORqWPcqxOOJvh1JkiRJSqhOBTg33XQTRx99NFlZWRQWFnLBBRewfPnyfX7m3nvvJRQK7dZSU13nQuotxvbP5t7PTue+K49h4sAcapta+PGz7zHzljnc/8Z6WlrbgilT06+Cry6EWTdD3jBoroWF/wf3zAoWPn7pJ1BXmujbkSRJkqSE6FSA8+KLL3L11Vfz2muvMXv2bGKxGKeffjp1dXX7/Fx2djZbtmxpb+vWrftQRUvqeY4bWcCjVx/PbZ88kuL8NEpqmvjWQ4s443/n8uu5q9lW3QjRNDj2i/CVt+CzT8GRn4JoRrAV+XPfh5+Og0e+BFveTvTtSJIkSVKX6tQixk8//fRuz++9914KCwt58803OfHEE/f6uVAoRFFR0YFVKOmwEQ6HOP/IgcyaUMT/vbaenz+/glXb6/jBk0v54VNLmTG8DxccOZBZE4vIHnIcDDkOzvyfYAereb+BzQtg4R+DNnhGsHPVoKMhfziEQh+8YDwOtSWQmh2EQ5IkSZLUQ32oXaiqqqoAyM/P3+dxtbW1DBkyhLa2NqZOncoPf/hDxo8fv9fjm5qaaGpqan9eXV0NQCwWIxaLfZiSE2Zn3T21fnWO/b1vYeDyYwbxscn9eOydrfzt7S28ub6SV1aV8cqqMr772Lt86pjB/OsJQ8lLT4EJ/wLjP0Fo85uE591FaOljhNa/CutfBSCekk28aCLxosmQmkuofBWUrSBUtpJQUzXx5Axaz7uD+OizDsn92N+9i/3du9jfvYv93bvY372Hfd279MT+7mitoXj8wFYHbWtr47zzzqOyspKXX355r8e9+uqrrFixgkmTJlFVVcWPf/xj5s6dy+LFixk0aNAeP/O9732PG2644QOv33fffaSnpx9IuZK6ubJGeLM0xJulYbY2BKNpUiJxTu4fZ2b/NtLeFzenxioYWvo8hdWLyG7YQCTesT/wlvS/mBX9ztnzaB1JkiRJSoD6+nouvfRSqqqqyM7O3utxBxzgfPGLX+Spp57i5Zdf3msQsyexWIyxY8dyySWXcOONN+7xmD2NwCkuLqa0tHSfN9OdxWIxZs+ezWmnnUY0Gk10OTrE7O8DF4/HmfNeKbf+fSVLt9YAkJsW5fPHD+HiaQPpk5my+wdaY1C6nNCWtwltfZtQcy3x/JHEC0YRzx8JucWEX/hvIvPvBqBtwsW0nn0rJB28xdTt797F/u5d7O/exf7uXezv3sO+7l16Yn9XV1dTUFCw3wDngKZQXXPNNTz++OPMnTu3U+ENQDQaZcqUKaxcuXKvx6SkpJCSkvKB16PRaI/pgL05HO5BHWd/H5jTJwzgo+P689S7W/np7OWs2l7HT/6+ktueX8VHx/bjE0cP4sRRfUmKhCEahUFTgrY35/wE+o2FJ/+D8LsPEK5cC5+8DzILD2rd9nfvYn/3LvZ372J/9y72d+9hX/cuPam/O1pnpwKceDzOl7/8ZR5++GHmzJnDsGHDOl1Ya2srixYt4qyzDs1aFJIOD+FwiLMn9WfWhCIeeWsTv391LW9vrOLpxVt5evFWCrNS+NiUgZw2rh9TBucRCe9nWtTRV0L+CHjgM7BxHtx+FIw5B8aeC8NPhujBG5EjSZIkSQdbpwKcq6++mvvuu49HH32UrKwstm7dCkBOTg5pacEOL5dffjkDBw7kpptuAuD73/8+xx57LCNHjqSyspJbbrmFdevWceWVVx7kW5F0OIqEQ1w4bRAXThvEsq3VPDB/Iw+/tYmSmibunLuaO+euJjc9yswj+nLK2H6cdERfctL2kmCPOBmufB7uvxRKl+/a0So5E0adDpM/GTy6Ro4kSZKkbqZTAc4dd9wBwMyZM3d7/Z577uGKK64AYP369YTD4fb3KioquOqqq9i6dSt5eXlMmzaNV155hXHjxn24yiX1OmOKsvnOOeP45qwxPL9sG0+9u5U5y7dTWR/jkYWbeWThZlKSwlw0bRBXnjCcYQUZHzxJwUj44iuw4TVY8hgs/RvUbIbFDwVt6Alwxg+h/6Suv0FJkiRJ2otOT6Hanzlz5uz2/NZbb+XWW2/tVFGStC/JSWFmTejPrAn9aWltY8H6Sp5bto3nlpawsqSWP76+nvveWM/p4/rxrycOZ9qQ/N1PEEmCoR8J2qybYfNb8O5fYf5vYO1LcOeJMPXTcMp3Dvo6OZIkSZJ0IA5oEWNJ6i6SImGmD8tn+rB8vjVrDK+vKefXc1fz3LISnlm8jWcWb2N43wwG5qZRkJlC36wU+mamMGFgDscOzycUDsOgaUE79gvw9+8FYc6C38O7D8P0q2D0mTBwGoQjib5dSZIkSb2UAY6kw0YoFOLY4X04dngfVmyr4e6X1vDwW5tYvb2O1dvrPnD88SP7cP2ZY5kwMCd4IXcwXPRbmP6v8PT1sHkBvPzToKXmBmvojDwNjjgDMgq69uYkSZIk9WoGOJIOS6P6ZfE/F03iulmjWbK5mu01TZTWNrG9poktVY3MXrKNf6ws49zbX+ZjUwbyjdNHMyA3WIydwcfClc/B0kdhyaOw6nlorITFDwctHIUxZ8O0K2DYSYm8TUmSJEm9hAGOpMNaQWYKJx7R9wOvbyiv55ZnlvPY25t5aMEmnnhnC5dMH8zZk/ozdXAekXAYxn8saK0tsOlNWDkb3nsGtr4DSx4JWt5QwpM/RWpzny6/N0mSJEm9hwGOpF6pOD+dn10yhc9/ZBg/eHIpb6wp595X1nLvK2vpk5HMR8f247Rx/fjIqAJSo0kw+JignfJt2LoI3vwdvPNnqFhLZM5/cwYQ33YHjPwojDgVhhwHyemJvk1JkiRJhwkDHEm92uTiXP78r8cy573tPLZwM88t3UZZXTN/nr+BP8/fQH5GMleeMIzLZwwlM2XHH5lFE+HsH8NpN8DiR2hb8HtCG14nVPoelL4Hr/0SIikw7EQYdx6MPhsyHKEjSZIk6cAZ4Ejq9UKhECePLuTk0YXEWtt4Y005s5ds45nFW9lS1ciPnl7Or+eu5soThnP5jCFkpUaDDyZnwJTLaJ3wCWY/9gCnj0ohae0cWPk8VG8MplytnA2hr8KQ42Hc+TD8ZMgfDuFwQu9ZkiRJUs9igCNJ7xONhDl+ZAHHjyzg22eP5dGFm7n9hZWsKa3jlmeWc9fc1Xzq2MFccORARvXLav9cLCmD+NizYNKFEI/D9mWw7HFY8liwZs7al4IGkJIN/ScHbcAUyCmG1BxIyw12u4qmJuTeJUmSJHVfBjiStBdJkTAXThvEBVMG8re3N/Pz51ewansdv3hhFb94YRVjirI478gBnDmucPcPhkJQODZoJ14H5WuCMGfZE7D5LWiq3j3Q+cCF06B4Oky8GMaeGwQ7kiRJkno1AxxJ2o9IOMQFUwZy7uQBPLN4Kw8t2MiL721n2dYalj29nB89vZxhWRHK8tdz7pGD6JuVsvsJ8ofBcV8OWmtLMDpny8IgzNnyDtSVQGNV0OJt0NIAa14M2hPXwsjTYOKFMOoMSMlMyHcgSZIkKbEMcCSpgyLhEGdN7M9ZE/tTWd/M0+9u5dGFm3ltTRlrakJ8/4ll/PeTyzh+ZAHnTh7ArAlFZO9cL6f9JElQNCFoUz61+3ttbdBcCzVbghE7ix6EkiWw/ImgRZKDtXSOmAVHnB6spSNJkiSpVzDAkaQDkJuezCenD+aT0wezsayGnzzwAqta8nhnYzUvrSjlpRWlfPfRdzlrQn8uPqqYY4blEw6H9n3ScBhSs4PWdzSc8HXYtgTefRDefQgq1sDqF4L29DehzygY/zGY/EnoM6JrblySJElSQhjgSNKH1C87lZn94/zorGPZVNXM397ezKNvb2ZlSS0PvbWJh97axOD8dC6eNogTj+jL8L4Zu3ay2u/Jx0G/78Ip34HSFbDiGXjvGVj/KpStgLk/Ctqgo2HSv8CECyE9/9DesCRJkqQuZ4AjSQfR0IIMvnzqKK45ZSQLN1Tyl/kb+dvbm1lfXs9PZr/HT2a/B0DfrBSGFWQwom8GRw3J55QxheRlJO/9xKEQ9D0iaMd9OVgv571n4Z0/w6rnYOO8oD19PQycGiyCXHwsFB8DmX276O4lSZIkHSoGOJJ0CIRCIaYMzmPK4Dy+c85Ynn53Kw+/tYllW2vYXtPU3t5YU86f3thAOARHDc3ntLH9OG1cP4YWZOz7Aqk5MOnioNVsC6ZZvf0n2LoINrweNH4eHJs3DHKLIaPvjlYAGYW7P88shOT9XFOSJElSwhjgSNIhlp6cxMenDuLjUwcBUN0YY21pHau317F8Ww1zlm9n6ZZq3lhTzhtryvnBk0spyExmVGEWR/TL5IiiLI7ol8XEgTmkRiMfvEBWP5hxddDKV8P612HDa8Hj9qXB2jkVa/ZfaEoOTPoEzPiSCyRLkiRJ3YwBjiR1sezUKJMG5TJpUC4A35w1hg3l9Ty3dBuzl27j9dXllNY2U1pbxqury9o/l5mSxKljCzlzQn9OOqIvacl7CHPyhwftyEuC5w0VwVbltSVQtz3YsrxuO9SV7njcDrXbg63Lm6pg3q9h3t0w9txgqlbx9C74RiRJkiTtjwGOJHUDxfnpXHH8MK44fhj1zS2sLKll+dYaVux4XLy5mtLaJh5duJlHF24mLRrhlDGFzJpQxCljCslI2csf52l5MPykfV88HofmOtg0H179Bax4FpY+FrRB04OdrkadBn1GBmvxSJIkSepyBjiS1M2kJyftNkIHoK0tzlsbKnhq0VaeencrmyobeGLRFp5YtIWUpDAnHdGXMycWcerYfmR3dIernUIhSMmE4TODVrIUXr0d3vkLbHwjaM9cD3lDYdTpMPKjwa5X7nYlSZIkdRkDHEnqAcLhENOG5DNtSD7/7+yxLNpUxZOLtvLUu1tYV1bPs0u28eySbUQjIU4Y1ZdZE4o4fVw/ctP3sbPV3hSOhfN/EWxd/u5fgxE5a/8BFWvhjbuCBsGInEFHw8BpwWO/8RDpZHgkSZIkqUMMcCSphwmFQu0jdL45azRLt9Tw1LtbeOrdrawsqeX5ZSU8v6yE/wyHmDGiD7MmFDFzdCEDc9M6d6Gsol2LIzfVwpoXgzBn9YvBoshlK4P29p+C45NSof+RMOiooBVNCnbLSs6EpBSnX0mSJEkfggGOJPVgoVCIcQOyGTcgm6+fPpoV22p46t2tPLloC8u21vDSilJeWlEKwNA+6Rw3soDjRxRwzPB88tOTCYc7GKqkZMKYs4MGUFcGm94M1s3ZOC/4ubEq2P1qw2sf/Hw4KQhysopgxKlwxOkw+DhI2jFCqK0NNr8Fyx6H5U8Gx59+I4w45SB8S5IkSVLPZ4AjSYeRUf2yGNUvi6+cOoo1pXU89e4WZi/Zxjsbq1hbVs/asvXc9/r69uNTksKkRiOkRsNkp0aZObov500eyISB2YT2NWImo08QwhxxevC8rQ3KV8HGnYHOfChdAbH6He+3QGNl0LYvg9d+AclZMOJkyCiA5U9Dzebdr/GHj8H0f4WP3gDJ6Qf1e5IkSZJ6GgMcSTpMDSvI4EszR/KlmSOpbozx+upy/rGylFdWlfLetloAmlraaGppo6oBtlU3saKkll+/tIbhBRmcO3kA5x05gBF9M/d/sXAYCkYFbecW5gBtrdBcG0zBaqoJwpsVzwatbnuw09VOyZnBbldjzoH1rwVbmr9xF6x6AT5+Z7DWDkBrC5Svhu1LIaMvDJ7h9CxJkiQd9gxwJKkXyE6Nctq4fpw2rh8A9c0t1De30hhrpTHWRmOslQ3l9Ty+aAt/X7KN1aV13PbcCm57bgUTBmZz/uSBnDO5P/1zOrmOTjgSrIOTmhM8LxwD4y/YNWVqxTPQUBHsbDXsJIimBsdNvAhGz4JHroayFXD3aXDELKhcD6XLobV51zVGnAJn3BScW5IkSTpMGeBIUi+UnpxEevLu/wmYMDCHMyf2p7aphdlLtvLows28tKKUdzdV8+6man741FKmD83nvCMHcNaE/uRlHMAOVzuFwzBoWtD2ZuRH4UuvwhPXwuKHYfkTu96LpgejfUqWwqrn4Y7jYPpVcNI33d5ckiRJhyUDHEnSbjJTkvjYlEF8bMogymqbePLdrfxt4WbeWFvO62uC9l+PLuakI/py3pED+OjYfmSkHKL/nKTnw0X3wORLoGQJFIwOtjnPHRKEQOWr4dnvBIsfv/4reOfPMP3foGgi9B0DeUMh4n/qJEmS1PP5f7WSpL3qk5nCp48dwqePHcKmygYef3szjy7czJIt1Ty3rITnlpWQFo0wc3RfRhZmUpyfzuD8dIrz0ynKTiXS0V2u9iUUgiPOCNo/yx8On/xjsE7O09cH6+K8ePOu9yPJkD8C+oyA3MGQUwy5xcFjwShIzvjw9UmSJEldwABHktQhA3PT+LeTRvBvJ41gZUkNjy3czGNvb2ZtWT1Pvbv1A8enRSNMG5LHscPzOXZ4HyYNyiU5KXxoihtxMnzhZXj7PljzUrBOzvb3oKUhCHW2L/3gZ8JJMGAqDP1I0IqPCbZLlyRJkrohAxxJUqeNLMzi2tNH8++nHcE7G6t4dXUZG8rrWV9ez4byejZWNNAQa+XllaW8vLIUgNRomKmD85g4KIcJA3KYODCHIX3S971deWdEkmDq5UGDYKHkqg1Q+h5UrA0WQK5cH7xWsQ7qS2HjG0F7+adBoFM4DvqND1rhOOg3ATIL3eVKkiRJCWeAI0k6YKFQiMnFuUwuzt3t9da2OCtLanl9TRmvrS7jtdXllNc188qqMl5ZVdZ+XFZqUhDmDMph/IBsJg7MYWifDMIHY+pVOAx5Q4K2JxXrYO3LO9pLQbCz9Z2gvV/2QBhyHAw5Phipk72X80mSJEmHkAGOJOmgi4RDjC7KYnRRFpfPGEpbW5wVJbUsWF/Bu5uqeHdTFUu31lDT2MKrq8t4dfWuUCczJYnhfTPITAl2yspIiZCenMSQPulccORAinJSD06RO8OdKZcFzyvXw5Z3YNtiKFkM25ZA+Sqo3gSLHggakJRRyPHkEan/M2QUQHqfYLHlaFowiicchUg0WH9n4LRgzR1JkiTpQzLAkSQdcuH3BTo7xVrbeG9bDYs3VbNoUxWLNlWxdEs1tU0tvLOxao/n+dHTyzh5dCGfnD6Yk0f3JSlyENfUyR0ctLHn7HqtuQ42zod1/4B1r8DGeYTqSiigBJYv79h5Bx8HEy+CcRdARp+DV68kSZJ6FQMcSVJCRCNhxg/IYfyAHD5xdDBKpaW1jZXba9lU0UBtUwv1za3UNbUEI3VWlfHG2vL23a8Ks1I4/8gBHD+ygKOH5h+arcyTM2D4SUEDaGmiZcN83nrxSaaOGUykqQrqy4LW0gitLdDWAm0xaKyGzW/B+leC9tR/wIhTYNDRO8KiHSOAMouC6V6SJEnSPhjgSJK6jaRImDFF2Ywpyv7Ae/9+GqwsqeXP89bz1wWbKKlp4tcvreHXL60hKRziyOJcjhvRh2OH92Fyce6hCXSSUogPms7mvFKOnHYWkWh038dXbYLFD8E7fwnW1lnxbNDeL5ISTLUafhIMnxn8HNnPeSVJktTrGOBIknqMkYWZ/L+zx3HdGWP4+9JtvLCshFdWlbGpsoH56yqYv66Cnz2/knAIxhRlM3VI7o6tzPvQPyet6wvOGQjHfTlo25fDsiegbBVUrgta1SZobdo1SmfOTZCcCYNnQP4wyCgMdsHKLITU3GDnrOotwbo81ZuhsQr6T4ZhJ0LxdEhK6fp7lCRJUpcwwJEk9TjJSWHOmtifsyb2Jx6Ps6G8gVdWlfLKqjLeXFfBpsoGlmypZsmWav7vtfUAHFmcy1kTizhzQn+K89O7vui+o4P2fq0twRbna1+C1XNgzVxoKIeVszt+3hXPwNwfQVIaDD42CHP6Tw62Qs/s5xbokiRJhwkDHElSjxYKhRjcJ53BfQbzyemDAdha1ciC9RW8uWNUzjsbK1m4IWg/fHIZEwZmc+ywPgzMS2Ngblr7Y05alFBXBh6RJCgYGbSjPgttbbBtEax/HWq2QF0J1G4PHhsqgx2vsgcEW5tn94doOmx4HVa/GByz+oWg7ZSWHwQ5RRN3bYOeltt19ydJkqSDxgBHknTYKcpJbR+hA1BS3cgzi7fy5KKtvL6mjHc3VfPupuoPfK4gM4VJg3KYODCHycU5TBqUS0FmF05LCoeD0TP9J3f8M9Ovgngcti8Lgpz1r+zaAr2hPBjds/YleO2XEArDgCnBWjtDjoOsAcFW6Gn5QZgkSZKkbsv/W5MkHfYKs1P59IyhfHrGUMpqm/j70m28t62WzZUNbKpsYFNFA2V1zZTWNvH8shKeX1bS/tlhBRmcMKqAE0b1ZcaIPqR0xw2jQiEoHBu0Y78QvBZrCNbd2bYYNi8Iwp2yFbDpzaC99JP3nyAYmZNRCHlDg/V38obteswb4vo6kiRJCWaAI0nqVfpkpvAvRw/+wOv1zS0s21rDOxsqeWdTFe9srGLV9lrWlNaxprSO37+6jqRwiCmDcylsDVG8qYojB/chHO6ma8xE02DAkUGbclnwWtUmWPNisN7O5regbjs0VADx4LGhAkqX7+FkoWDaVv6wIODpMwIKx0O/ccHrrrMjSZJ0yBngSJIEpCcnMXVwHlMH57W/Vt0Y47VVZcxdsZ2XVpSyrqyeeWsrgAhP/Op18tKjfGRUX04YVcDoflkUZKVQkJlMSlIkcTeyLzkD4chLg7ZTa0sw1aq+LFh3p3wNVKzZ8bg2eIzVQfXGoK19afdzpuZA4Y4gJ9YAzTXQXBe0aHqwPfqIU6H4GEhK7tLblSRJOpwY4EiStBfZqVFOH1/E6eOLAFhXVscLy7bx15cXs7o+SkV9jL+9vZm/vb35nz6XREFWCilJEcKhYIBKOBQiKRziIyML+NSxQyjMTk3ELX1QJGnXVuWFY2HEP70fjwcjdXYGOhVroPS9YJ2dshXBVubrX937+TcvgJdvhWgGDDshWHunz0jIHx6M5ol2YHv3xqpg2ldyJgw8KlgrSJIkqZcxwJEkqYOG9MngsunF5JUu4rQzTmbx1jrmvredV1aVsbmygdLaJmKtcaobW6hubNnjORasr+SOF1dx7qQBfO4jw5gwMKeL76KTQqFdAc/gY3Z/r6UJSlcE6+zUbYfkjCBkSckMfq7eDKueD1rddnjv6aC9X/bAIMjJGfS+Vgz15bDhtWBHrpIlQDw4Pqs/jDsfxn8MBk03zJEkSb2GAY4kSQcgGglz9NB8jh6az9d3vBaPx6lqiFFa20RpbTOx1jba4tAWjxOPxymvi3H/G+uZv66Ch97axENvbWL60HxOGt2X0f2yGF2UxcDctO67rs4/S0qBoglB25vJn9yxPfq7sOo52PIOlK8ORvQ0VUH1pqDtT95QqK8Ipnm9/qugZQ2AUR+FAVOD3bUKxzlNS5IkHbYMcCRJOkhCoRC56cnkpiczsnDPx1w0bRBvb6jknn+s4fF3tvDG2nLeWFve/n5GcoQjirKYUpzH0UPzmDY0j8KsbjLd6kCFw9B/UtB2iseDUTblq6ByPVRt3L1FU4N1c3a2rH7BiJ9VL8Dih2H5k1CzGRb8PmgAkWToNx76jvngiJ70AkjNhkg0Md+BJEnSh2SAI0lSF5tcnMv/fnIK3zpzLI8u3MSSLdUs31rDqu211DW38tb6St5aX8lv/7EGgKF90pk2JJ9Jg3IYNyCbMUVZZKX28CAiFIKMPkErnt6xzySlwOhZQdsZ5mx4PdhRa/Nb0Fi56+e9niMtCHJSsoNRPf3GQ78JwWPBKAMeSZLUbRngSJKUIEU5qfzbSbtWDY61trG2tI7Fm6t5c10F89aWs3xbDWvL6llbVs9fF2xsP3ZIn3SO6JdFVkoSSZEQ0UiYaCRMenKEycW5TB+aT17GYTyd6P1hDgQjeirWBuFNxdoPjuhpqgqOa2mA2gao3RYswrxy9q5zhqOQkgXhCITCEIqQFI5wfGs64afnBFPFCsdB4RiIpEBLI7Q2B2FSWwtkFQVr/0iSJB0CBjiSJHUT0UiYUf2yGNUviwumDASgqiHGgvUVLFhXwZLN1SzZUs2WqkbWldWzrqx+n+cbU5TFMcPyOWZ4H6YPy6cgM6UrbiMxQiHIHxa0PWltgabqoDVWQUMllK0MFmDe2Zprgi3V339aoADgzeUdqyOzKNhhq8/w4HFnyxsWjPyRJEk6QAY4kiR1YzlpUU4eXcjJo3ctqlNe18zSLdWs2l5LU6yN5tY2YjtaeV0z89ZWsLKklmVba1i2tYbfvboOgJGFmRw7PJ9jhvXhmOH5PX9tnc6IJEF6ftB2Gn7Srp/j8WCkTnMdxFsh3gZtrbQ0N/D2Cw9z5IBkItuXQclSqN74vhOHICk1GLETq4ParUFb/8oHa8joG0zbSssPRvqkZAaPyZnBeeJtQDyoJZwE/cYFCzRnDwgCKkmS1KsZ4EiS1MPkZyRz/MgCjh9ZsNdjSmubeGNNOa+vLuP1NeUs21rDypJaVpbU8n+vrQeCETonjCrgxCP6cvTQfFKjka66he4nFILc4g+8HI/F2JhfwqRTziIS3bE+TnNdELIkpQRBy85wpb4cKtYEO2yVrw5a2argsb402Eq9bnvna8vsF+yyVTQJ0vJ2bdOevCMEat+6fcfzpFQDH0mSDkMGOJIkHYYKMlM4a2J/zprYHwhG7byxppzX15Tx+upylm6tbh+h8+uX1pCSFGbakDwG5aXRLzuVwuxUirJTyc9IJiUpWF8nKRIiORImMyWJ3PQood4aEuxtnZudI3wGTvvge41VQbBTuQ4aq6GpZkerDgIh2LHuTggIQaw+2HK9ZEmwXs97TwetI6LpQeAz6CgYNB0GHR3s4iVJkno0AxxJknqB/IxkZk0oYtaEIiAIdF5eWcpL721n7ortbKtu4pVVZR0+X256lBF9MxnRN4MRfTMZ1S+Tcf1z6Jed0nuDnX1JzYEBRwatM5rrYdu7sGkBbF8WhD7NtdBUG6zZ01S763lsRxAUq4d1/wjaTln9g0WWMwohs2/wmFUUTM/KHhhst57RF9paoXQ5bF4IWxYGj20tMOp0GHM2FE10dI8kSQligCNJUi+Un5HMeZMHcN7kAcTjcVaU1PL2hkq2VTeyrbopeKxporyuiZbWOLHWNppb2oi1xmmItVJZH+PNdRW8ua5it/MWZCYzbkAO4wdkM2lgDtOH5dPncF48+VBLTg+2We/IVuttrcFonurNsHHerlayFGq2BG1fIilBONPS+MH3Ni+AF2+G3MEw5hwYegJkFkJ6n6ClZBnsSJJ0iBngSJLUy4VCIY7ol8UR/bI6dHxDcytrSutYtb2WVduDdXWWb61h1fZaSmubmfvedua+t2utl1GFmRw7PFg4+djhfQ7v3bASKRwJdrpKzQ62Op/66eD1xmoofQ9qS6CuBGq3B481W6F6E1RtCqZptTYFxydnQf9J0P/IYCpWazMsfxJWPgeV6+G1Xwbt/SLJO8KcAsjos+vn9D47nu/8ecdjWn6wsPQ/i8eD6WZ124PpZbFGiDUE27+3NAU7eg2Y8uHDongcqjYEo46iaR/uXJIkdREDHEmS1ClpyRHGDchm3IDdt8VuaG5l2dZqFm8O2lvrK1i2tYYVJbWsKKnlD6+tIxSCyYNyOXVMIaeMLWRc/2ynXB1qqdnBejj70tIMNZuDUTx5wyAc3v39KZcF07lWPQ/LngjW5qkvC1qsPgh5OjLK5/3S8nYFPa3NQWhTW7IrSNqbrAEw+kwYcxYMPRGSkjt+zbY2WPY3ePlW2PxWcO3p/wpHXxkETZIkdWOdCnBuuukmHnroIZYtW0ZaWhrHHXcc//M//8Po0aP3+bkHHniA73znO6xdu5ZRo0bxP//zP5x11lkfqnBJktS9pCVHmDI4jymD89pfe//iya+uKmPZ1hoWbqhk4YZKfjL7PYqyU5k5ui8zRvRhxvA+FGb3oq3Nu5Ok5GCL831JToex5wTt/Zrrd4Q5pcFj3ft/Lt0V9Ox83lABxIPHhgooW/nBa6VkBy2aBtFUSEqDSDRYk6dmM8z/TdCSs6Dv6GDx6LT8XY9ZRcG6PjmDgjV+wkmw6C/w8v9C2Ypd16kvhTk/DAKdKZ+Co/9t399BfXkQXpWvCRaz3jmiKL0guHYkuv/vWpKkA9SpAOfFF1/k6quv5uijj6alpYX//M//5PTTT2fJkiVkZOx5R4ZXXnmFSy65hJtuuolzzjmH++67jwsuuIAFCxYwYcKEg3ITkiSpe/rnxZO3VTfywrIS/r60hJdXbmdrdSP3z9vA/fM2ADC8bwYzhvfhqKF5jCrMYmRhZu/e3rwnSE4P2h62Yd+jttYguGkPd0qDgCWzXzClKbNw79OaYo2wZi4sfwKWPxVM/do0f//XTErdtbZPas6OUTdXwdqX4B+3wdZ3YN6vSZp3N6dHc0nafEtQR2ZhsL5P+WrYtiQIj/YmFIHhM2HiRcE6QanZez9WkqQD0KkA5+mnd9++8t5776WwsJA333yTE088cY+fue2225g1axbXXXcdADfeeCOzZ8/m9ttv51e/+tUBli1JknqiftmpfHL6YD45fTCNsVZeXV3GP1aU8urqMpZsqWb19jpWb6/jj6+vB4KlTgbnpzOqMJPs1CjVjS3UNsWobWqhtrGF4vx0zp00gDMmFJGT5uiHHiEcCUauZBR0/rPRVDji9KCdfStsfTtYw6ehPAiF6suDUKhmC1RtDFpzbRDeZPaDGVfDtM/uClcmXgQTLoQ1L8I/fkZo1XOkxSpgWwVs20sNOYOhYGQQJrWPPCqHeCusei5oSf8OR5wB484PdvyKJAejc5JSgsWiUzKDYCia7uLPkrQntduhYk2wi2L2wA9O7W1pgi1vw4Y3YOMbcOyXYPCxiam1C32oNXCqqqoAyM/P3+sxr776Ktdee+1ur51xxhk88sgje/1MU1MTTU275j9XV1cDEIvFiMViH6LixNlZd0+tX51jf/cu9nfvYn8fPBHgI8Pz+MjwPGAUlfUx5q2t4LU15SzZUs3KkjoqG2KsK6tnXVn9Hs+xtqyel1aU8v8eWcTMI/py7qQiTjqigPTkg7PMn/3dzfWdELS9iceDxZBrS4IdtJJ2LKD9z/1ZfDx88nhiFZuY99zDHDNhJElN5YTqSqGxEnIHEy8cR7zv2CB4+WdtrVCxhvCSRwgvfpBQ2UpY8mjQ9iEeCkNyZjAdKxwNwq1wUtBCoWAB51hDEEDFGoK1gkLh4L1QOGjRNOK5QyFvKPG84cTzhwUjmVpjwfE7WqixEqo2EaoOgq1Q9SYIhYkPPIp48XTig6YTL5q86zva+f21NAbhU/jwWzrT39+9xyHv63gboQ2vQc1W4iNODUb67U3ttmAR+fzhe/7z5GBoawl+74eTgj9bDiQobo0R2raI0MY3CG2aD4010BYLzt3WGoTWmf2I548g3mck5I8gnj88+HMpVg/N9YRidcGfXfFWiAPxNiAe/Nnyzz+3NhEqWUJo6zuEti4iVLu1vZR4UhrkDw+ulVEQvL/1bUKtzbvKLRhLW/9pQM/8vd3RWkPxeDx+IBdoa2vjvPPOo7KykpdffnmvxyUnJ/O73/2OSy65pP21X/7yl9xwww1s27bnf9r43ve+xw033PCB1++77z7S09MPpFxJktTDxONQE4NtDSG2NkCsDVIju1pyJM6amhDzt4fZ2rDrf05DxClKhyGZcQZnxhmcESc5AnUxqGsJURuD+hbokwpH5MRJP/z+XqpEisfJaVjHwIrX6FuzhHA8RrgtRjjeQjjeSqStmaS2RkIc0P+CH1KtoSiN0TwibU0ktTURaWsiRJy2UIT65L7UpvSjLqUfdSlFtIRTSWprIKm1kaS2RpJaG2gNR2mK5tKYlBM8RnNpDSdDPE7wOzS45+akLFoi+979KxRvIa25grTmUtKbS0lrLiMtVkFdSl/KMkZTmT6MeA8LlbIaNjCs9HnSmkvZmDeDzXnHEA85RZR4nKTWekK0ESdMPBQhHtr52L2/n/Sm7RSXv0xx+ctkNAe7L7aEktmcdzTr+sykPOMICIVIaq1nQOV8BpW/QkHt0vbf/w3RPGpSB1Cb0p/a1CLqk/tSn1xAQ3JB+++RSGsj6c3byWjeTnpTKdBGQ3IfGqJ9aEjOpykpm2hrHfl1K3e0FeTWrSYpvivciBOiLRShJZJGU1I2TUnZNCdl0ZSUTVt499GroXgr2Q0byKtfRVJbM4kSJ0RjNI+UlirC8dY9HtOUlEV5xkgq0keyLWcy1WmDu7jKg6e+vp5LL72UqqoqsrP3PgX3gAOcL37xizz11FO8/PLLDBo0aK/HHUiAs6cROMXFxZSWlu7zZrqzWCzG7NmzOe2004hGHeJ9uLO/exf7u3exv7un5Vtr+Ns7W3li0RY2VjZ2+HORcIgpxTmcMLKAk44oYGxRFuHwrjDI/u5duqy/4/HgX6ibaqC5BprrCLW17viX7Z3/ut0WjK5JSg2mWkV3LOQc3/Ev1rS1jzAKVawlVLEaytcQqlxLqL6c+PunbYWjkJJFPGcQZA8inj2AePYgQrH64F/XN84LHutLD909//NXkFEY/Gt93jDiecMg1kCoekMwOqhqA9RsIRRv2/vnk9KID5xGvPiYYCHp3YSIp2Tv2Oksn3haXrC4dTQ9mM62YzTCfvu7tRlqthKqLyOenBEsmp2yY8QUIagvDWqt2hiMbGqoIp47GPqMDEYkpPeBlkZCSx8jvOB3hDe+vvs95BTTdsyXaJt86Y5zEhy/7V1CmxcG0/OyiohnDSCePRCyBwQjtpprg5FlTTWEmmqIR9OgcFxwbx9GvA2a64LzvH8k1sFSu43QxvmEtizY1cc1W4PHlj3/uR3fMcKjffRF3nBIzyMezdjRF5kQTiJUtoLQtiVQsjgYxVG5jnj2gKAv8kfSkjuM15ZvYfoxxxINxYORJK2xXSPVdo5Y2fn7L5wEkaTgu9gZFDbVEGqshqaq4PfdloWE1/1jV63JmZBVFIzA2/lan1HE+4wktOp5Qu/bYS+elk+ooXyfX1c8LQ9CYUL1Zfs+Lhwl1HZoRprEU3OD0XnFxxDPLHrfKMFgVE+oeiOUrSJUvpJQ2argObzvz630YOprOCkYmUMoaKGgxdt/DgfH5I8kXjSJeNFE4v3GB/3b1gKV6wmVrSRUvgpqtwUjIgcdDblD9zi6qCf+t7u6upqCgoL9BjgHFFtfc801PP7448ydO3ef4Q1AUVHRB4Kabdu2UVRUtNfPpKSkkJLywT80otFoj+mAvTkc7kEdZ3/3LvZ372J/dy8TivOZUJzP9WePY2tVI29vrOSdjZW8vaGKRZuqiMfj5Gckk5eRTH56MlmpSSzaVMWq7XXMX1fJ/HWV3PrcSgqzUjhtXD9OG9ePGSP6tPex/d27dEl/JydDRu7BOdfAyR94qcMTJoZ/JHiMx4PFmutKg0Wpo+lBqBBND8KCslVQvio4pmw1tDQEf7lKyd4VasQag+khO1vNtmAKVuh9f2kDiNUTqishVFcCG17be22R5B27iRUHi2RnFsH2ZbD+VUL1ZYTWvQzr9j4TYI9C4WBXs2gqSUlpnNLUStrmWwglpweLXYeToK4EqrcEAcqeTxKEY637GZ2QlheEIo3BshOEk2DM2VBwBMy/h1DVBiLPXk/kpR/ByI8G91ayNPgLa2dFUqD/JBh4FAycBjkDg4BwZ+DQWB304/4ed44MS8nZsVZV3+AxFArO9/4WTtr16yQ5Y8d0wPT3/ZwR1LV9WbBGSeX6Tt9WqHYbodptsP6Vzn+2rgS2LAy+HuAkgOWdPs3+rgLDT4IjLyM05pwgaN04HxbcC+8+FARLO3e+6zsGJl4MEy8ilDc0WLerdAVsXw6ly4Pd7SrXQ9UGaKgg1FCx6zKpuZA3BHKHBL+GqzcF63/VbNkV3vQZBcXHQPH0YC2Y7IG7AqmdgVVTNdRtD36f15UGv8b39Os4bxgMnkGo4AhC/7z2zL60NEM4QijcsZFTHftzKgr9Rgetk3rSf7s7WmenApx4PM6Xv/xlHn74YebMmcOwYcP2+5kZM2bw3HPP8bWvfa39tdmzZzNjxozOXFqSJKlDinJSKcop4ozxe//Hop02lNczd8V2Xly+nX+sLKWkpok/vr6eP76+nsyUJE4c1QeqQlTN20DfrDTyM5Lpk5nCoLw0d8fS4SUUgj4jgvbPUrODIGX4SQfnWg2VQRDU3tYEf/HNLQ7WKsoZHPycUfjBhUshCJtK34N1/wj+shxr+Kf3d4QmDeVQXxE8Ntfuei9WB7E6QkAWwLYte681khxsE9/SEIQc8VYgvusvvZlFQa05xcG6JxVrgrBrx1/CAcgeBNOugKmfDra4Bzjh67DwPnjl58FnFj2w65rpBTBwajDipmZb8Jf16s27B0pJacH6KSlZuxbx3jgvaAdDU1XQylcdnPMBEApGCg06CgpG7VicdkDwmNU/CMXev75KSxNUrgu+z7KVQatYG/RDcy001e7o13jwHfcbv6vlDwu+s9IVULqCttL3aN62kpS0dEJJycEIkkjy+0bZRIPrR5KDgKStZfdROvG24PdBak7QUrKDmsed/8Ed+IqPDtoZN8Hih4I+HH0mFE3cfbRIWl4QthRP38P3XxOEOW2tQXCztzV1WmPBou3RDMj455FoCZD0IUeBab86FeBcffXV3HfffTz66KNkZWWxdWuwsFBOTg5pacEcvcsvv5yBAwdy0003AfDVr36Vk046iZ/85CecffbZ3H///cyfP5+77rrrIN+KJElS5xTnp3PZMUO47JghNLW08uqqMmYv2cbsJdsoqWniyXe3ARGe3LB0t8+FQlCcl87IwkxGFmYyvCCD3PRk0pMjpCdHSEuOkJmSxIDcNKKRTvzrpdQbpOUGAcXAqQf2+VAI+o4O2lGf69hnWpqDaWstjcFjrJGWxhpee3kOx06bTFI8Fowgam0ORp1k94esAZCev+sv3fF4EBY11UBrU7Cz2d6mGjXXB+FUc10QWPzziIRoGhz9+SDYWfZ4sE19v3EwYGoQlu1p0dnYjtpTsoPgYaedo6c2vRkEWpveDAKd1OwdI6SydoUOO1/758f3/9zaFIzOqC3ZMVpje1BPSvaOUVc7ppLtnHK1x1bbvpAtuYODQGPgtH0v7gu7f08pWcHon4HT9n58fEeYtqd+eN/nWmMxnnnySc4666yuG5GRmh3074FIyQqCqP2JRIPvV71GpwKcO+64A4CZM2fu9vo999zDFVdcAcD69esJvy8pP+6447jvvvv49re/zX/+538yatQoHnnkESZM2MeOAZIkSV0sJSnCzNGFzBxdyI3nT+DtjZW8sHQb8xavICO/HxUNLZTXNbO9ponaphbWl9ezvrye55eV7PWcyZEwIwozGVOUxZiiLI4oymJwfjoDctJIS3YEj9RlkpI/MDogHotRlrlj16CO/KU+FNoxRagDm6okp0NRB/6+E44EozjGnb//Y6OpQdtTXTtHT036xP7Ps1/pweiQglEH4VyHWCh0aNbrkbqpTk+h2p85c+Z84LWLL76Yiy++uDOXkiRJSphwOMSUwXlM6J/Jk43LOeusKe3/ahuPxymtbWZlSS0rt9eyqqSWNaV11DTGqG9upSHWSn1zKzWNMRpjbSzdUs3SLdUfuEZeepQBuWkMykvjuBEFnDq2kEF57rYpSZL2rGftvSdJkpRgoVCIvlkp9M1KYcaIva85EI/H2VjRwLKtNSzfWs3SrTWs2FbDpooG6ppbqaiPUVEfY/Hmap5ZvI3/emwxY4qy+OjYfpw8ppBR/TLJTu0Ziy9KkqRDzwBHkiTpEAiFQhTnp1Ocn85p4/q1vx6Px6lubGFLVQObKxt4b1stzy8rYf7acpZtrWHZ1hpufyHYhjY7NYni/HQG5aXRPyeNpHCIcDhEKAThUIi0aITJxblMG5JHZor/WydJ0uHM/9JLkiR1oVAoRE5alJy0KGOKsjllTD++cNIIKuqamfNeCX9fWsLrq8sorW2murGFxZurWbz5g1Ow3i8SDjFhQDbHDO/DlOJcCrJSyEmLkpsWJTst6o5ZkiQdBgxwJEmSuoG8jGQ+NmUQH5syCIC6phY2VTawsaKeDeUNbK1upK0tThzaH8vrmpm3tpyNFQ28vbGKtzdW7fHcuelRjh9ZwCmjC5k5ui99Ml30U5KknsYAR5IkqRvKSEniiH5ZHNEva7/HbqpsYN6acl5fU8bSLTVUNcSorG+mqiFGWxwq62M88c4WnnhnC6EQTB6Uy0dGFjCyMJMhfdIZtmMbdEmS1H0Z4EiSJPVwA3PTGDhlIBdMGbjb621tcWqbW1ixrYYXlm3n+WUlLNlSzcINlSzcULnbsTlpUYb2SWdoQQZD+mQwrCCdIX0yKMpOJSUpTDQpTHIkaOFwqAvvTpIkgQGOJEnSYSscDpGdGmXakHymDcnnG2eMZmtVI3OWl/DW+krWltWxrqyerdWNVDXE9jkN6/0KMlM4YVQBJx3Rl4+MKqDAKVmSJB1yBjiSJEm9SFFOKp+cPphPTh/c/lpDcyvryutYW1q/I9SpY01p8Ly0tomWtvhu5yitbeLhtzbx8FubAJg4MIfjRvRh3IBsxg/IYVhBBhFH6UiSdFAZ4EiSJPVyackRxhRlM6Yoe4/vt7XFibW1EWuN09zSxnvbanjxve3MfW87izdXs2hTFYs27Rq5kxaNMKZ/FmOKshicn8HQPukM7hNMyXK7c0mSDoz/BZUkSdI+hcMhUsIRUpKAFDh2eB+OHd6Hb84aQ0lNIy+vKOWt9ZUs3lzF0i01NMRaeWt9JW+tr/zAuQoykxnSJ4Mh+UGgM6RPOpkpScRa22hubaO5JQiKBuSmcvTQfDIMfCRJAgxwJEmS9CEUZqXy8amD+PjUYPvz1rY4a0rrWLy5itXbg+lY68rrWVdWT3ldM6W1QXtzXcV+z50UDjFlcC4zRhRw/Ig+jOqXRU5a1OlZkqReyQBHkiRJB00kHGJkYSYjCzM/8F51Y4z1ZUGYs7asjvU7Hhtb2kiOhEhOChONhEkKh1i6pSbYHn1tBfPWVvCz51YAEApBdmqUvPQoeRnJTB6Uy7mTBzB1cC6hkMGOJOnwZYAjSZKkLpGdGmXCwBwmDMzZ77HxeJwN5Q28sqqUf6wq47XVZWyvaSIeh6qGGFUNMdaW1fPW+krufWUtg/LSOHfyAM6bPICBeWk0xlppirXR1NJK414eI+Fwe3CUHImQFGqjubULvghJkg6AAY4kSZK6nVAoxOA+6Qzus2vHrFhrG5X1MSrrm6moj1FS08hzS0t4dvFWNlY0cMecVdwxZ9WHum5KJMKrsXe5aFoxxwzv43QtSVK3YYAjSZKkHiEaCdM3K4W+WSntr50zaQANza08t2wbjy3czJzl22lubSMUgpSkMKnRyB4fU5IitLbFaW5tCxZQbmmjrLaJ7bXNPPTWZh56azNF2amcf+QAJg7KYWBuGgPz0uibmeJULUlSQhjgSJIkqUdLS45wzqQBnDNpALHWNtricZIj4U4HLU1NzfziL09Rkj6UJxdtZWt1I3fOXb3bMclJYQbkpJKVGiU9ORK0lCQyk5MYkJvG4D5pDM5Ppzg/3bBHknRQGeBIkiTpsBGNhA/4s+FwiBHZ8OWzxnHD+RN4Ydl2Zi/ZxvryOjZVNLC1upHmljbWltV36HzpyREmDMxh6uA8pg7OZcrgvN1GD0mS1BkGOJIkSdI/SUmKMGtCEbMmFLW/FmttY2tVI5srG6hrbqG+uZX6plbqm1uoamhhU2U968vr2VDewJaqBuqbW3ljTTlvrClvP8fg/HSOHZ7PjBF9mDG8gKKc1ETcniSpBzLAkSRJkjogGglTvGN61P40t7SxrqyOt9ZXsmB9BW+tr+S9khrWlwchz1/mbwRgeEEGxwzvw5TiXCYMzGFUv8wPNYpIknT4MsCRJEmSDrLkpDCj+mUxql8Wnzi6GIDqxhgL1lXw6uoyXl1VxrubqlhdWsfq0jr+9MZ6IFh4eWz/bMb2z6ZPRjK56VFy0qLkpieTnxFlUF6wtk7Y3bEkqdcxwJEkSZK6QHZqlJmjC5k5uhCAqoYYb6wpZ97achZt/P/t3Xl0lPX9L/D3M/tMMjNZJzNDSAgBEkJIWkAi+gNFUQi4Vdu6YItL6YZWxaq1p269vdUrt9XbHqu9xwVrq6e1R/RX6gYCan9EoMHIJgFC9sxkn32f+d4/AvO7YxIIS5bJvF/n5CTzPN/nmc+TT76Zmc/5fr+PEwfanXAHI6hrdaCu1THseVRyGaZkapGfqYXVqEV2ugpZaSrkpKuRlaZCpk4FvUYBvUaBdI0CaoV8jK6QiIhGEws4RERERETjwKhV4oqyPFxRlgcAiMUEmvt82NfmwPFuL5z+MBy+EBz+MBy+MLrdwYGFlKMxNPZ40djjHdHzqBQyZOlUKMjSoSBbh8IT34tz0zErTw+VglO2iIiSAQs4REREREQTgEwmoSgnDUU5acO2iURjsDkDaOv3o63fB7szgF5vCL3eEPq8QfR6Quj3heAJROANRQEMrMdjdwVgdwWwu6kv4XxKuYQSsx5zpxhRPsWIUrMeUzN1yNXzFuhERBMNCzhERERERElCkbCQcvYp20ZjAp5gBJ5gBN3uIJp7vWjp9aG5z4eWXh/qO91w+sM40O7CgXYXgNb4sWqFDPmZWuRn6lCUk4ZZeXqUmNMxM08Pg0Y5uhdJRERDYgGHiIiIiGgSksskGLUDiyBPydDia1MzEvYLIdDW78f+dif2tw+swXO82wub049gJIaGbi8aur34+Eh3wnFWowbTctJgMWphzdDAbNTAatTGp2cpeBctIqJRwQIOEREREVEKkiQpPppn5VxLfHs4GoPNEUBrvw+tfT4c7/Gi3u7GkU43bM4AOk58DUUll6EoJw0z8tIx05QOtUIOhz8Epy+Mfl8ITn8YFqMWC4uyUFWUhaKcNE7VIiIaIRZwiIiIiIgoTimXoSB7YKHjr3L6wzjS6UZrnw82ZwA2px92ZwDtjgCaerzwh6Oo73SjvtN9yufY9Hk7ACBXr0ZVURZK8vSwZGhhNWpgydDCYtRAozz93bOEECwAEVHKYAGHiIiIiIhGxKhV4oJpWbhgWtagfbGYQIfTj6OdHhztcuNYlwfRGJCpUyJDp4RRp4JBo0BDlwefNfahrtWBbncQm/fZsBm2QefL1Cnj07QsRi3yDGq4AxG0O/wDI4EcfnS7gyizGlBdbkF1uRnTTrEANBFRsmMBh4iIiIiIzplMJiE/U4f8TB2WlppO2z4QjuKLVgf2NPWhudcHu2ugKGNzBuALRdHvC6PfF8Yhm+uU59nX5sS+Nif+1/uHUWYxYOVcM5aV5aEkTz+uo3NcgTAOdbiwoDCT6wIR0XnBAg4REREREY05jVKOqunZqJqeeDctIQRc/ghsLj9sjgA6nAPfO10B6DVKWDM0sGZoYc3QIkOrxM6GXry734aa4704ZHPhkM2F//3hEViNGlxaasLSEhMunpENnWrsPvo09XjxnZd3obXPj2nZOvzwkmJcPy8fKgULOUR09ljAISIiIiKiCUOSJBh1Shh1SpSaDadtPy0nDbdUFaDPG8KWQ3a8f8COnQ296HAG8PquFry+qwUKmQStUg5IgEySIJMG7tKll+TYGT6E2RYDZpn1mGnSIztNBZns7Efu7G9z4rZXdqPXGwIANPX68LO39uP/fHQU318yHTddUACt6vTr+xARfRULOERERERElPSy0lS48YIC3HhBAQLhKGoaerG9vgvbDnehrd8PdzAy6JgeSGj8d1vCNpkEZOhUyEpTIUunQna6CjPz9CizGFBmMWBqlnbYqVmfHu3GD1+rhTcUxRyrAX9YPQ9bDnXi/35yHDZnAE/84xB+99FRrCg3o7rcgkXF2VByehURjRALOERERERENKlolHIsLTVhaakJT1wjYHcFEAzHEBMCAgPTtDz+EN7a+l9Is8zAsR4fjnS60dLnQ0wAfd4Q+k6MoAGA9w7Y4z/r1QqUWgYKOrNPfJWY9fjwUCfu/1sdwlGBi2dk44Vb50OvUeJ7i6fj1gsL8ffaNrzwcQPa+v14Y3cr3tjdCqNWiSvK8lBdbsZ/zMyBWsGROUQ0PBZwiIiIiIho0pIkCRajdtD2cDiM5hyBlVfMhFKpBACEIjE4fCH0+QYKOP3eMOyuAA6fWFvnaKcH7mAEe5r6saepP34umQTExMDPqyos+O23KxOKMRqlHLdeWIibLpiKmuO9eO+AHR8etKPHE8Lfa9vw99o26NUKXDbbhOpyMy6ZZeI0KyIahAUcIiIiIiIiACqFDCaDBiaDZsj94WgMDd0efGlz4VCHC1/a3Dhkc8VH69x20TQ8elXZsGvoKOQyLJ6Zi8Uzc/E/ri3HnqY+vH/AjvcO2NDpCuKdug68U9cBrVKORcXZKMjSwWI8uWjzwHeTXgP5OazRQ0TJiwUcIiIiIiKiEVDKZSg1G1BqNuAbXx/YJoRAlzuIUCSGqVm6EZ9LLpNw4fRsXDg9G49eVYbPWx14/4AN7+63o93hx7bDXcMeZzZoYM3QwGLUYqYpHVfOMWNWXvq43jadiEYfCzhERERERERnSZIk5A0zYmekZDIJ8wszMb8wEz9fORsH2l2oa3PA5vCjw+FHx4nbqdudAURiAu0OP9odfgAD07h+s+UIpuekYUW5GSvKzZg7xchiDtEkxAIOERERERHRBCFJEubmGzE33zhoXzQm0O0OosM5UNhp7/djd2MfPj3ag+M9XvxhRwP+sKMBOpUcBo0Seo0C6RoF0tUKSJIEbzACbzACdyACXyiCTJ0K804UjhYUZqI4Nz1h+lckGoM3GIVOLefdsogmABZwiIiIiIiIkoBcJsFs1MBs1GBeQSYA4AeXFMMdCGN7fTfeP2DD9sPd8IWi8IWisLtOfb5+XxjHe7z4e+3ArdSNWiWy01XwBCLwBCPwhaIAAIVMwrScNMzITcfMvHTMMKXDoFEiGhOIiYEvIYDZFgOm5aSN6u+AKJWxgENERERERJTE9Bolrqm04ppKKwLhKOzOADzBCFyBMDyBgRE3AJCmHhiNk6aWI02tQHu/H7XN/fh3cx/qWh1w+sNw+sODzh+JCRzr8uBYlwfvHzx1LPMKMnD9vHxcVWFBhk512th9oQja+v3ITlMhK03FqV9Ep8ACDhERERER0SShUcpHPApmVp4eS0tNAAbusHXY5oYvFEG6RgG9emAKlk4tR68nhGNdHhw9UcRp6PIgEIlCkiTIJEAuSQhHY9jf7sTeFgf2tjjwy38cwmWlpvjiynLZQFsA6HAG0NjtRWOPF3ZXIB5Phk6J4tx0FOemnfiejmJTOqZmaqHgFC4iFnCIiIiIiIhSnVIuG3LdHQAnbmOuxZJZuac8R5crgHfqOvDW5+340ubC+wftpx2xAwB6jQKeYAQOXxi1zf2obe5P2K+SyzAtR4fi3HTMzTdifkEmKvIzoFXJR3x9RJMBCzhERERERER0zkwGDdYumY61S6bjUIcL7x+wweEPn1gnB4idWDPHbNCgKDcNRTnpKMpOg1GnRCAcRWOPd2CET7cHDd1eNHR5cLzHg0A4hiOdHhzp9OC9A3YAA+vylFkNmFeQicUzc3BRcQ4LOjTpsYBDRERERERE51WZ1YAyq2HE7TVKOWZbDJhtSTwmFhPocPrR0O3FEbsbn7cOjNDpdAWxr82JfW1ObNzZBLVChkXF2bis1ISLZ+RACMDpD8HhC6PXHcAeuwTDsV7MtmYgz6DmWjuUlFjAISIiIiIioglJJpOQn6lDfqYOl5yYwiWEQIczgNrmfuxu7MX2w91od/ixo74bO+q7hzmTHG821gIA9GoFZuSlo9Ssx+KZuVg8Mwd6jXKMrojo7LGAQ0RERERERElDkiRMydBiSoYW11RaIYTAkU4Pth3uwvbDXahrdUCjlCFDp0KGTgmDRgFHbzd8cj2a+3xwByP4vMWBz1sceGN3K5RyCRdOz8blpSYsKs5BNCbgDUXit1MPR2PISVfDZFAjT69Bhk45aASPEAKeYAStfX609fvQ2u9Ha58PAHBpSS4uKs6BSsGFmOncsIBDRERERERESUuSJJSY9Sgx6/GjS4sH7Q+Hw3j33XexcuXFEJIcTb1eHOl0o67FgW2Hu3C8x4tPj/bg06M9I3o+lVwGo06JSDSGcFQgFI0hHI1BiKHbb9zZBL1agaWlJiyfY8aSWRzxQ2eHBRwiIiIiIiJKCSqFDLPy9JiVp8dVFVb84qoyNHR78NGXndj6ZRcOdbigUcqh1yiQrlYgTS2HQiZDtzuILncA/b4wQtEYut3BIc+flabC1EztiWlfWrgCEWz9shPd7iD+84sO/OcXHQCAXL0a07J1KMhKw7RsHWbm6bGwKAtZaaqx/HVQkmEBh4iIiIiIiFJWcW46inPT8f0lg0fvfFUwEkW3OwiXPwKVQoJSLoNSLoNKIYNOJYdONfgj9v+MlePzVgc+PGjHBwftaOr1odsdRLc7iD1NibdMLzXrceH0bFw4PRtfm5oBk14NmYwLLtMAFnCIiIiIiIiIRkCtkCM/UwdkjvwYmUzC/MJMzC/MxMMrZ8PpC6O5z4umXh+aewa+72934EinB4ftbhy2u7FxZxMAQCmXYDZqYDEOrPmjVcnhD0XhC0XgC0XhD0VhNmqwcq4FS0tMw95K3ROMQAiBdLWCd+BKYizgEBEREREREY0Ro06JCl0GKvIzErb3eILYdbwPnx3vRc3xXjT2eBGOCrT2+dHa5z/lOTfvs0GrlOPy2SZcVWGBQaPE/nYn9rc7caDdiabegQWVtUo5cvVq5OrVMOnVmFeQiW/Mm4KcdPVoXS6dRyzgEBEREREREY2znHQ1VlVYsKrCAgCIRGPocgfR4fCjwxlAe78fwUgUOpUcWpUCOqUcGqUc+9od+Oc+G9r6/di8z4bN+2zDPoc/HEVLnw8tJ+6Q9d4BO57+4DCuKMvDtxdMxeKZuZCfmLIlhIArEEGPJwi5JA2sC6RRQK0YepQPjT4WcIiIiIiIiIgmGIVcBmuGFtYM7Snbraqw4GcrSrGvzYl/7rfhg4N2RGMCc6cYUT7FGP+uUZ5cjHlg/Z32fj8277fhi1YH3t1vx7v77bAaNTAbNfE2wUhs0POpFDLo1QqoFDLIJAkKuQT5ie/5mTrMNKVjhikdM/P0mGFKR7qaZYfzhb9JIiIiIiIioiQmSRIqp2agcmoGfr5y9rDtCrMVKMxOiz9eu2Q6vrS58Nc9rdj0eTs6nAF0OAMJx+jVCsSEgDcUBQCEIjH0RkJDnv9IpwfbDnclbJuekzYQW74RlVMzUGY1cBTPWWIBh4iIiIiIiChFzbYY8Pg1c/Cz6lJ8cqQb0ZiAyaCGSa9Brl4NjXKg2BKNCXiCEbgDYXiCEYQjAlEhEI3FEI0N3KGrudeHY10eHO1y42inB13uII73eHG8x4tNn7cDGFiY+etTM3HRjGxcPCMHlfkZUClkAACnL4xDNhcOdjjR1u/HHKsBi2fmwmzUjNvvZyJhAYeIiIiIiIgoxWmUclw5xzzsfrlMglGrhFGrHLbN4pmJj/u8Iexrc+CLVifqWvvxRZsTfd4Qdjf1YXdTH57dehQ6lRzlU4zocPjR1j/0Ys0zTOn4jxk5WFScjcJsHcwGDYxaZcrdUYsFHCIiIiIiIiI677LSVLi0xIRLS0wABhZGbur1oaahF//V0IOaht6Bgk5jX/yY/Ewt5lgNmJKhw96Wfuxrc+BYlwfHujzx26sDA2vx5BnUMBs0uPXCQlz7tSljfXlj7owLOJ988gk2bNiA2tpa2Gw2bNq0Cdddd92w7Xfs2IGlS5cO2m6z2WA2D1/dIyIiIiIiIqLJQ5IkFOWkoSgnDbdUFSAWEzhsd+NghxP5mTqUWQww6hJH+Dh9YdQc78GnR3tQ29yPTlcA/b4wQpFY/Bbrq+ZaxumKxtYZF3C8Xi8qKytxxx134Prrrx/xcfX19TAYDPHHJpPpTJ+aiIiIiIiIiCYJmUxCmdWAMqth2DZGnRIryi1YUf7fRZpAOIpudxCdrgDsrgDmWI1jEe64O+MCTnV1Naqrq8/4iUwmEzIyMkbUNhgMIhgMxh+7XC4AQDgcRjgcPuPnnghOxp2s8dOZYb5TC/OdWpjv1MJ8pxbmO7Uw36mDuZ585ADMeiXMeiUqp+gBDM5zMuV7pLFKQghxtk8iSdKIp1AVFhYiGAyivLwcjz/+OC6++OJhj3n88cfxxBNPDNr++uuvQ6fTnW24REREREREREQTis/nwy233AKn05kwc+mrRr2AU19fjx07dmDBggUIBoN48cUX8dprr2HXrl2YN2/ekMcMNQJn6tSp6OnpOeXFTGThcBhbtmzBFVdcAaVy+FW7aXJgvlML851amO/UwnynFuY7tTDfqYO5Ti3JmG+Xy4WcnJzTFnBG/S5UJSUlKCkpiT++6KKL0NDQgGeeeQavvfbakMeo1Wqo1epB25VKZdIkYDiT4Rpo5Jjv1MJ8pxbmO7Uw36mF+U4tzHfqYK5TSzLle6RxykY5jiEtXLgQx44dG4+nJiIiIiIiIiJKOuNSwKmrq4PFkhq3+SIiIiIiIiIiOldnPIXK4/EkjJ5pbGxEXV0dsrKyUFBQgIcffhjt7e3405/+BAB49tlnUVRUhDlz5iAQCODFF1/Etm3b8OGHH56/qyAiIiIiIiIimsTOuIDz73//G0uXLo0/Xr9+PQBgzZo12LhxI2w2G1paWuL7Q6EQ7r//frS3t0On06GiogJbt25NOAcREREREREREQ3vjAs4l156KU5146qNGzcmPH7wwQfx4IMPnnFgREREREREREQ0YFzWwCEiIiIiIiIiopFjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJjAYeIiIiIiIiIaIJTjHcAIyGEAAC4XK5xjuTshcNh+Hw+uFwuKJXK8Q6HRhnznVqY79TCfKcW5ju1MN+phflOHcx1aknGfJ+sdZysfQwnKQo4brcbADB16tRxjoSIiIiIiIiI6Pxzu90wGo3D7pfE6Uo8E0AsFkNHRwf0ej0kSRrvcM6Ky+XC1KlT0draCoPBMN7h0ChjvlML851amO/UwnynFuY7tTDfqYO5Ti3JmG8hBNxuN6xWK2Sy4Ve6SYoRODKZDPn5+eMdxnlhMBiS5o+Izh3znVqY79TCfKcW5ju1MN+phflOHcx1akm2fJ9q5M1JXMSYiIiIiIiIiGiCYwGHiIiIiIiIiGiCYwFnjKjVajz22GNQq9XjHQqNAeY7tTDfqYX5Ti3Md2phvlML8506mOvUMpnznRSLGBMRERERERERpTKOwCEiIiIiIiIimuBYwCEiIiIiIiIimuBYwCEiIiIiIiIimuBYwCEiIiIiIiIimuBYwCEiIiIiIiIimuBYwBkjzz33HKZNmwaNRoOqqirs3r17vEOic/Tkk0/iggsugF6vh8lkwnXXXYf6+vqENpdeeikkSUr4+uEPfzhOEdO5ePzxxwflsrS0NL4/EAhg3bp1yM7ORnp6Om644QZ0dnaOY8R0LqZNmzYo35IkYd26dQDYt5PdJ598gquvvhpWqxWSJOHtt99O2C+EwKOPPgqLxQKtVotly5bh6NGjCW36+vqwevVqGAwGZGRk4M4774TH4xnDq6CROlW+w+EwHnroIcydOxdpaWmwWq347ne/i46OjoRzDPU/4amnnhrjK6GROF3/vu222wblcsWKFQlt2L+Tx+nyPdRruSRJ2LBhQ7wN+3dyGMlnr5G8H29pacGqVaug0+lgMpnwwAMPIBKJjOWlnBMWcMbAX//6V6xfvx6PPfYY9u7di8rKSixfvhxdXV3jHRqdg48//hjr1q3DZ599hi1btiAcDuPKK6+E1+tNaLd27VrYbLb419NPPz1OEdO5mjNnTkIu//Wvf8X33XffffjHP/6BN998Ex9//DE6Ojpw/fXXj2O0dC727NmTkOstW7YAAL71rW/F27BvJy+v14vKyko899xzQ+5/+umn8bvf/Q4vvPACdu3ahbS0NCxfvhyBQCDeZvXq1Th48CC2bNmCzZs345NPPsH3v//9sboEOgOnyrfP58PevXvxyCOPYO/evXjrrbdQX1+Pa665ZlDbX/7ylwl9/u677x6L8OkMna5/A8CKFSsScvnGG28k7Gf/Th6ny/f/n2ebzYaXX34ZkiThhhtuSGjH/j3xjeSz1+nej0ejUaxatQqhUAg7d+7Eq6++io0bN+LRRx8dj0s6O4JG3cKFC8W6devij6PRqLBareLJJ58cx6jofOvq6hIAxMcffxzfdskll4h77rln/IKi8+axxx4TlZWVQ+5zOBxCqVSKN998M77tyy+/FABETU3NGEVIo+mee+4RxcXFIhaLCSHYtycTAGLTpk3xx7FYTJjNZrFhw4b4NofDIdRqtXjjjTeEEEIcOnRIABB79uyJt3nvvfeEJEmivb19zGKnM/fVfA9l9+7dAoBobm6ObyssLBTPPPPM6AZH591Q+V6zZo249tprhz2G/Tt5jaR/X3vtteKyyy5L2Mb+nZy++tlrJO/H3333XSGTyYTdbo+3ef7554XBYBDBYHBsL+AscQTOKAuFQqitrcWyZcvi22QyGZYtW4aamppxjIzON6fTCQDIyspK2P6Xv/wFOTk5KC8vx8MPPwyfzzce4dF5cPToUVitVkyfPh2rV69GS0sLAKC2thbhcDihn5eWlqKgoID9fBIIhUL485//jDvuuAOSJMW3s29PTo2NjbDb7Qn92Wg0oqqqKt6fa2pqkJGRgQULFsTbLFu2DDKZDLt27RrzmOn8cjqdkCQJGRkZCdufeuopZGdn4+tf/zo2bNiQVEPuKdGOHTtgMplQUlKCH/3oR+jt7Y3vY/+evDo7O/HPf/4Td95556B97N/J56ufvUbyfrympgZz585FXl5evM3y5cvhcrlw8ODBMYz+7CnGO4DJrqenB9FoNOGPBADy8vJw+PDhcYqKzrdYLIZ7770XF198McrLy+Pbb7nlFhQWFsJqtWLfvn146KGHUF9fj7feemsco6WzUVVVhY0bN6KkpAQ2mw1PPPEEFi9ejAMHDsBut0OlUg16s5+Xlwe73T4+AdN58/bbb8PhcOC2226Lb2PfnrxO9tmhXrdP7rPb7TCZTAn7FQoFsrKy2OeTXCAQwEMPPYSbb74ZBoMhvv0nP/kJ5s2bh6ysLOzcuRMPP/wwbDYbfvvb345jtHQ2VqxYgeuvvx5FRUVoaGjAz3/+c1RXV6OmpgZyuZz9exJ79dVXodfrB01xZ/9OPkN99hrJ+3G73T7k6/vJfcmABRyi82DdunU4cOBAwpooABLmS8+dOxcWiwWXX345GhoaUFxcPNZh0jmorq6O/1xRUYGqqioUFhbib3/7G7Ra7ThGRqPtpZdeQnV1NaxWa3wb+zbR5BMOh/Htb38bQgg8//zzCfvWr18f/7miogIqlQo/+MEP8OSTT0KtVo91qHQObrrppvjPc+fORUVFBYqLi7Fjxw5cfvnl4xgZjbaXX34Zq1evhkajSdjO/p18hvvslQo4hWqU5eTkQC6XD1r9urOzE2azeZyiovPprrvuwubNm7F9+3bk5+efsm1VVRUA4NixY2MRGo2ijIwMzJo1C8eOHYPZbEYoFILD4Uhow36e/Jqbm7F161Z873vfO2U79u3J42SfPdXrttlsHnQjgkgkgr6+Pvb5JHWyeNPc3IwtW7YkjL4ZSlVVFSKRCJqamsYmQBo106dPR05OTvz/N/v35PTpp5+ivr7+tK/nAPv3RDfcZ6+RvB83m81Dvr6f3JcMWMAZZSqVCvPnz8dHH30U3xaLxfDRRx9h0aJF4xgZnSshBO666y5s2rQJ27ZtQ1FR0WmPqaurAwBYLJZRjo5Gm8fjQUNDAywWC+bPnw+lUpnQz+vr69HS0sJ+nuReeeUVmEwmrFq16pTt2Lcnj6KiIpjN5oT+7HK5sGvXrnh/XrRoERwOB2pra+Nttm3bhlgsFi/mUfI4Wbw5evQotm7diuzs7NMeU1dXB5lMNmiqDSWftrY29Pb2xv9/s39PTi+99BLmz5+PysrK07Zl/56YTvfZayTvxxctWoT9+/cnFGlPFu3LysrG5kLOEadQjYH169djzZo1WLBgARYuXIhnn30WXq8Xt99++3iHRudg3bp1eP311/HOO+9Ar9fH500ajUZotVo0NDTg9ddfx8qVK5GdnY19+/bhvvvuw5IlS1BRUTHO0dOZ+ulPf4qrr74ahYWF6OjowGOPPQa5XI6bb74ZRqMRd955J9avX4+srCwYDAbcfffdWLRoES688MLxDp3OUiwWwyuvvII1a9ZAofjvl0v27eTn8XgSRks1Njairq4OWVlZKCgowL333otf/epXmDlzJoqKivDII4/AarXiuuuuAwDMnj0bK1aswNq1a/HCCy8gHA7jrrvuwk033ZQw1Y4mhlPl22Kx4Jvf/Cb27t2LzZs3IxqNxl/Ps7KyoFKpUFNTg127dmHp0qXQ6/WoqanBfffdh1tvvRWZmZnjdVk0jFPlOysrC0888QRuuOEGmM1mNDQ04MEHH8SMGTOwfPlyAOzfyeZ0/8+BgSL8m2++id/85jeDjmf/Th6n++w1kvfjV155JcrKyvCd73wHTz/9NOx2O37xi19g3bp1yTNdbpzvgpUyfv/734uCggKhUqnEwoULxWeffTbeIdE5AjDk1yuvvCKEEKKlpUUsWbJEZGVlCbVaLWbMmCEeeOAB4XQ6xzdwOis33nijsFgsQqVSiSlTpogbb7xRHDt2LL7f7/eLH//4xyIzM1PodDrxjW98Q9hstnGMmM7VBx98IACI+vr6hO3s28lv+/btQ/7/XrNmjRBi4FbijzzyiMjLyxNqtVpcfvnlg/4Oent7xc033yzS09OFwWAQt99+u3C73eNwNXQ6p8p3Y2PjsK/n27dvF0IIUVtbK6qqqoTRaBQajUbMnj1b/PrXvxaBQGB8L4yGdKp8+3w+ceWVV4rc3FyhVCpFYWGhWLt2bcIthYVg/04mp/t/LoQQf/zjH4VWqxUOh2PQ8ezfyeN0n72EGNn78aamJlFdXS20Wq3IyckR999/vwiHw2N8NWdPEkKIUawPERERERERERHROeIaOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREExwLOEREREREREREE9z/A4oD5N6C1rK8AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 1400x600 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"EPOCH = 200 # количество эпох обучения\n",
"history_train = [] # список значений лосса трейна на каждой эпохи\n",
"history_test = [] # список значений лосса теста на каждой эпохи\n",
"model.to(device) # И модель, и данные должны находиться на одном устройстве.\n",
" # Поэтому при работе с GPU нужно следить и явно указывать, на каком устройстве проводится работа.\n",
"\n",
"\n",
"best_test_loss = float(\"inf\")\n",
"for i in range(EPOCH): #цикл по эпохам\n",
"\n",
" loss_test = 0\n",
" loss_train = 0\n",
"\n",
" for batch in dataloader_train: #цикл по тренировачным батчам\n",
"\n",
" optimizer.zero_grad() #обнуляем градиенты\n",
" batch_ix = torch.tensor(batch[0], dtype=torch.int64).to(device) #делаем из батча тензор\n",
"\n",
" predictions_logp, _ = model(batch_ix[:, :-1], hid_state=None) #подаем батч в модель\n",
"\n",
" actual_next_tokens = batch_ix[:, 1:] # таргеры\n",
"\n",
" loss = loss_fn(predictions_logp.permute(0, 2, 1), actual_next_tokens.long()) # считаем лосс на батче\n",
" loss_train += loss.item() # добавляем лосс с батча в суммарный лосс\n",
"\n",
" loss.backward() # делаем обратный проход\n",
" optimizer.step() # делаем шаг оптимизатором\n",
"\n",
" history_train.append(loss_train/len(dataloader_train)) # добавляем средний лосс за эпоху в список\n",
"\n",
" for batch in dataloader_test: #цикл по тестовым батчам\n",
" with torch.no_grad(): # отключаем подсчет градиентов\n",
"\n",
" batch_ix = torch.tensor(batch[0], dtype=torch.int64).to(device)\n",
" predictions_logp, _ = model(batch_ix[:, :-1], hid_state=None)\n",
"\n",
" actual_next_tokens = batch_ix[:, 1:]\n",
"\n",
" loss = loss_fn(predictions_logp.permute(0, 2, 1), actual_next_tokens.long())\n",
" loss_test += loss.item()\n",
"\n",
" loss_test = loss_test/len(dataloader_test)\n",
" history_test.append(loss_test)\n",
"\n",
" if loss_test < best_test_loss: #сохраняем лучшую модель по лоссу на тесте\n",
" best_test_loss = loss_test\n",
" best_model = copy.deepcopy(model)\n",
" best_model.to(\"cpu\")\n",
"\n",
" if (i + 1) % 5 == 0: # выводим график лосса каждые 5 эпох\n",
" clear_output(True)\n",
" plt.plot(history_train, label='loss_train')\n",
" plt.plot(history_test, label='loss_test')\n",
" plt.grid()\n",
" plt.legend()\n",
" plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iWGz3oH3Rowq"
},
"source": [
"### Вопрос 8\n",
"Достаточно ли обучилась модель? Имеет ли смысл изменить количество эпох обучения?"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "k924KVVaGU4t",
"outputId": "01db571f-54df-421e-cdf5-6c6c4ce7bf72"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"perplexity (best model test): 7.181648032772434\n",
"perplexity (last epoch test): 7.421088521940808\n"
]
}
],
"source": [
"print(\"perplexity (best model test): \", np.exp(best_test_loss))\n",
"print(\"perplexity (last epoch test): \", np.exp(loss_test))"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5fwRbhzDGVxl",
"outputId": "230be0e7-f04c-4c63-8576-2e2971a82b00"
},
"outputs": [
{
"data": {
"text/plain": [
"CharGRULoop(\n",
" (emb): Embedding(143, 64)\n",
" (gru): GRU(64, 256, num_layers=3, batch_first=True)\n",
" (hid_to_logits): Linear(in_features=256, out_features=143, bias=True)\n",
")"
]
},
"execution_count": 52,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"device = 'cpu' # Обучение обычно проводится на GPU, но чтобы не тратить его ресурсы, работу по генерации текста уже обученной моделью стоит перенести обратно на CPU\n",
"model.to(device)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0PWo3arcGWvv"
},
"source": [
"Погенерируем текст. Сначала зададим стартовую фразу.\n",
"\n",
"Выполнив следующие ячейки по несколько раз, убедитесь, что правильно ответили на вопрос 7 \"Чем отличается стратегия greedy от sample?\""
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"id": "dtUW5hVBGX9p"
},
"outputs": [],
"source": [
"#Если вы не согласны, вы можете поменять стартовую фразу, но что думает об этом высказывание машина?\n",
"SEED_PHRASE = 'Теория автоматического управления - лучший предмет,'"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Nr24uuQRGZqQ"
},
"source": [
"Sample strategy"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "bY_iYC82GY28",
"outputId": "1a41d952-882b-451a-b321-f2066f03f255"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Теория автоматического управления - лучший предмет,\n",
"В вечерчят, может и ребких ряд,\n",
"Когда в тихает Наздним не рафстились взор,\n",
"Где боющей слус во знали цвю,\n",
"Еврожачи в пустыню твоей, прочей,\n",
"В Ажертрянный лес пропариду улещенье,\n",
"Истревняешь литее молви от меня за Eилей\n",
"В и ждох эти пыраматься спомнорой,\n",
"И вишел ее пеельчим на пытом.\n",
"\n",
"Зевители, моей пужко образ роды погоглавляек;\n",
"Скворный, сокружлен ускоре вином:\n",
"\n",
" EOS любится каз ним меня дороды!\n",
"Всях казы-тро ты сладо мною рошу.\n",
"Нет, и красота тебе прироща\n"
]
}
],
"source": [
"print(generate_sample(best_model, char2id, id2char, seed_phrase=SEED_PHRASE, strategy=\"sample\", max_length=MAXLEN))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "i6n4dUScGbk9"
},
"outputs": [],
"source": [
"#prompt = \"<не сдерживайте себя, сгенерируйте что-нибудь про соседа>\"\n",
"#print(generate_sample(model, char2id, id2char, seed_phrase=prompt, strategy=\"sample\", max_length=256))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyB9g3KOGe8k"
},
"source": [
"Greedy strategy:\n"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "HuTB1fdlGd0b",
"outputId": "83440df4-700d-4c3b-935b-8e3aa5f65bcb"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Теория автоматического управления - лучший предмет,\n",
"И в тебе под него не след не внемлет он мой под сон,\n",
"И в тебя с полей под сенью покой приветный страсть устались,\n",
"И в тебя в тебя с поле под сенью покой,\n",
"И с тобой страсть и страсть у него в поле молчаливой,\n",
"И с тобой в тебе с полей под страсть и страсть на волненье,\n",
"И в тебя в тебя с поле под страсть и страсть на восторга,\n",
"И с тебя с невольно прости страсть и страсть на восторга,\n",
"И с тебя с невольно прости страсть и страсть на восторга,\n",
"И с тебя с невольн\n"
]
}
],
"source": [
"print(generate_sample(best_model, char2id, id2char, seed_phrase=SEED_PHRASE, strategy=\"greedy\", max_length=MAXLEN))"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"id": "_tKKZheZGfu8"
},
"outputs": [],
"source": [
"#prompt = \"<не сдерживайте себя, сгенерируйте что-нибудь про соседа>\"\n",
"#print(generate_sample(model, char2id, id2char, seed_phrase=prompt, strategy=\"sample\", max_length=256))"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"id": "RpVEloyFScaa"
},
"outputs": [],
"source": [
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jxpdQXl1GhyL"
},
"source": [
"# Эксперименты с температурой"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LdKkJc5vGjIt"
},
"source": [
"В функции `generate_sample` есть параметр `temperature`.\n",
"\n",
"Основываясь на прошлом пункте, выберите стратегию, которая больше понравилась и запустите ячейки с разной температурой"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"id": "h88cuY0HGfsa"
},
"outputs": [],
"source": [
"nice_strategy = \"sample\" #<your choice here>"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "9uZZRmP-GlS2",
"outputId": "46f77170-a06e-4e72-98ca-f30d76e0c5f2"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Теория автоматического управления - лучший предмет,\n",
"Нет.У дух преБЦя! убиием?> Скутxщ:: —\n",
"мбебле) был за цев чещу\n",
"Па. ТДа чевтовых: а домите,\n",
"Хтон живлчивые Посов, я , куке, лесь,\n",
"<пучалу> лябесимlим КлOE<жуе оRлки-пом,\n",
"Бедпятные пъе! сrакиса: ебя.\n",
". нелаться, лены яхтну…\n",
"М зигранном дня, <судую,\n",
"Афием, П*\";c——\n",
"ДТрома, нуй, чтоко и; Дьбл\n",
"Еу! укнавнпиdусем.и.. ь,я\n",
"\n",
"Гдю Увер!ы\n",
"Дррикаявь, твое гофJ —!\n",
"Уе.ш. И, Бстоъ тиханью,\n",
"Лю??\n",
"Хласуя âлучит хожделz штон!…Л.\n",
"…»>\n",
"ожестгу<чиг\n",
"Ясь.…а хванни дупгухНвеннемляцы. —\n"
]
}
],
"source": [
"print(generate_sample(best_model, char2id, id2char, seed_phrase=SEED_PHRASE, strategy=nice_strategy, max_length=MAXLEN, temperature=2.0))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DwIORQ3OoiKN"
},
"source": [
"### Вопрос 9\n",
"Сделайте выводы как влияет изменение температуры на генерацию текста.\n",
"\n",
"Выберите оптимальное значение температуры"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"id": "4DoOXClVohes"
},
"outputs": [],
"source": [
"\n"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"id": "htAW_ABDGnc6"
},
"outputs": [],
"source": [
"# По завершению работы с рекуррентной сетью, очистим кэш\n",
"torch.cuda.empty_cache()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rlcgdw_gGpiO"
},
"source": [
"# Bonus track GPT"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FND3cMDMGra3"
},
"source": [
"Дальше происходит магия, чтобы все вышло:\n",
" - здесь лучше перезапустить сеанс (Среда выполнения -> Перезапустить сеанс)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "HEZqH-4rGo84",
"outputId": "881559c1-9659-4157-97fc-ccf9425169b3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m297.3/297.3 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.7/23.7 MB\u001b[0m \u001b[31m43.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m823.6/823.6 kB\u001b[0m \u001b[31m46.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.1/14.1 MB\u001b[0m \u001b[31m62.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m731.7/731.7 MB\u001b[0m \u001b[31m1.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m410.6/410.6 MB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.6/121.6 MB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 MB\u001b[0m \u001b[31m11.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m124.2/124.2 MB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m196.0/196.0 MB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.0/166.0 MB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m99.1/99.1 kB\u001b[0m \u001b[31m13.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.1/21.1 MB\u001b[0m \u001b[31m58.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h"
]
}
],
"source": [
"!pip install -q transformers[torch]"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"id": "5ccG2XtoGtDT"
},
"outputs": [],
"source": [
"import locale\n",
"import torch\n",
"import transformers\n",
"import numpy as np\n",
"\n",
"from warnings import simplefilter\n",
"from IPython.display import clear_output\n",
"from transformers import Trainer, TrainingArguments\n",
"from transformers import GPT2LMHeadModel, GPT2Tokenizer\n",
"from transformers import TextDataset, DataCollatorForLanguageModeling\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "xerwrDTGGtAc"
},
"outputs": [],
"source": [
"# Задаем некоторые настроечные параметры касательно кодировки и отображения предупреждений\n",
"locale.getpreferredencoding = lambda: \"UTF-8\"\n",
"simplefilter(\"ignore\", category=FutureWarning)\n",
"transformers.logging.set_verbosity_error()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "ygm4wl7EGvLX"
},
"outputs": [],
"source": [
"\n",
"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
"\n",
"model_name = \"sberbank-ai/rugpt3small_based_on_gpt2\" # Опередлим, какой моделью будем пользоваться\n",
"tokenizer = GPT2Tokenizer.from_pretrained(model_name) # Определим токенайзер для нашего текста\n",
"model = GPT2LMHeadModel.from_pretrained(model_name).to(device) # Загрузим предобученную модель трансформера rugpt3small от Сбера\n",
"\n",
"clear_output()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "jfAXAgtfGvIs",
"outputId": "a9f47a68-7a4a-4edc-be98-cbf539e277cb"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Теория автоматического управления - лучший предмет, который я знаю.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
}
],
"source": [
"SEED_PHRASE = 'Теория автоматического управления - лучший предмет,'\n",
"input_ids = tokenizer.encode(SEED_PHRASE, return_tensors=\"pt\").to(device)\n",
"out = model.generate(input_ids, do_sample=False, max_length=20)\n",
"\n",
"generated_text = list(map(tokenizer.decode, out))[0]\n",
"\n",
"print(generated_text) #Так работает предобученный трансформер"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "By_ukdhOp_F8"
},
"source": [
"Давайте дообучим трансформер на нашем датасете - мы хотим генерировать стихи"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "YLIYVZ1AGxzV"
},
"outputs": [],
"source": [
"train_path = \"train_dataset.txt\"\n",
"\n",
"with open(\"poems.txt\", encoding=\"utf-8\") as file:\n",
" data = file.read().split(\"</s>\\n\\n\")\n",
"\n",
"with open(train_path, mode=\"w\", encoding=\"utf-8\") as f:\n",
" f.write(\"\".join(data))"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "ulVcdoROGyy2"
},
"outputs": [],
"source": [
"train_dataset = TextDataset(tokenizer=tokenizer, file_path=train_path, block_size=128) # Создание датасета\n",
"data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False) # Создание даталодера (нарезает текст на оптимальные по длине куски)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "nTkPLolOGzqH"
},
"outputs": [],
"source": [
"\n",
"training_args = TrainingArguments(\n",
" output_dir=\"./finetuned\",\n",
" overwrite_output_dir=True,\n",
" num_train_epochs=15,\n",
" per_device_train_batch_size=32,\n",
" per_device_eval_batch_size=32,\n",
" warmup_steps=10, # рекомендованные значения (warmup_steps нужен для \"разогрева\" сети, c его помощью learning rate постепенно увеличивается до заданного значения)\n",
" gradient_accumulation_steps=16, # рекомендованные значения\n",
")\n",
"#(обычно мы хотим положить батч по-больше, чтобы сеть побыстрей сошлась, но мы ограничены памятью gpu, gradient_accumulation_steps накапливает (суммирует или усредняет) градиенты за прогон на 16 батчах )\n",
"\n",
"trainer = Trainer(\n",
" model=model,\n",
" args=training_args,\n",
" data_collator=data_collator,\n",
" train_dataset=train_dataset,\n",
" optimizers=(\n",
" torch.optim.AdamW(model.parameters(), lr=1e-5), # рекомендованные значения\n",
" None,\n",
" ),\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MN4IUz0cG06R"
},
"source": [
"Эта ячейка займет около 15-20 минут"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MIDjczLBG1x4",
"outputId": "4cd93a79-50a4-4f67-dc0d-1736a9055395"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'train_runtime': 512.7748, 'train_samples_per_second': 43.499, 'train_steps_per_second': 0.059, 'train_loss': 4.064910634358724, 'epoch': 10.21}\n"
]
}
],
"source": [
"output = trainer.train() # Дообучаем трансформер на наши тексты"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "jGSF1YRpHzUw",
"outputId": "76ecac12-dd33-4005-d1bd-4a6e27a0f74e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"perplexity: 58.25970187428917\n"
]
}
],
"source": [
"print('perplexity: ', np.exp(output.training_loss)) #расчет перплексии"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CssQW8rLHzUx"
},
"source": [
"### Вопрос 10\n",
"Какое значение перплексии получилось у трансформера?\n",
"\n",
"Какое значение перплексии получалось у рекуррентной сети?\n",
"\n",
"Почему у рекуррентной сети значение было существенно ниже, но качество текстов хуже? Почему нельзя сравнивать значения для рекуррентной сети и трансформера?\n"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ZEG2FYNsG4oy",
"outputId": "0bad01e8-fb2a-4a34-c1ba-39dd01017a66"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Теория автоматического управления - лучший предмет,\n",
"Когда в нем все подчинено закону. \n",
"\n",
"И только законы в нем\n",
"Ошибки и воры\n",
"Так искусно ведут против\n",
"Нимфы его дела;\n",
"Они как раз в ней с самого начала\n",
"Узрели: человек есть душа,\n",
"Коли вдруг он не становится ни грешным,\n",
"Ни преступником,\n",
"Ни преступником.\n",
"\n",
"Он как будто в ней живет,\n",
"И в ней дух живет,\n",
"В ней свет есть для всех он,\n",
"Она его питает,\n",
"Она о нем поет.\n",
"\n",
"Но для него ничего не значит\n",
"Свой день, свои жертвы,\n",
"Он не может быть свободен,\n",
"И он ни над кем не властен:\n",
"В нем весь ход земного рая\n",
"В нем его предел.\n",
"\n",
"Он хочет жить, он хочет умереть,\n",
"Готов он ко всему и не может\n",
"Предать себя сомненьем;\n",
"В нем человек в конце концов\n",
"Перестает жить и для себя.\n",
"Ведь он же человек,\n",
"Он ведь даже в своем законе\n",
"Омертвляет грех силою мысли.\n",
"\n",
"Но все напрасно: с нею человек живет:\n",
"И в нем живет она со своей душой.\n",
"Но\n"
]
}
],
"source": [
"SEED_PHRASE = 'Теория автоматического управления - лучший предмет,'\n",
"input_ids = tokenizer.encode(SEED_PHRASE, return_tensors=\"pt\").to(device)\n",
"model.eval()\n",
"with torch.no_grad():\n",
" out = model.generate(\n",
" input_ids,\n",
" do_sample=True, # sample strategy\n",
" temperature=1.0,\n",
" max_length=256,\n",
" pad_token_id=512 # указываем id <PAD> токена\n",
" )\n",
"\n",
"generated_text = list(map(tokenizer.decode, out))[0]\n",
"print()\n",
"print(generated_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kK91Afxmqsnr"
},
"source": [
"### Вопрос 11\n",
"Проверьте работу ячейки выше для разных стартовых фраз и разных параметров `temperature`, `max_length`, `do_sample` и объясните, за что отвечает каждый из параметров. Подберите (субъективно) лучшие"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "URWWyeN_G92m"
},
"source": [
"# Beam Search\n",
"До этого мы использовали обычный \"жадный\" поиск\n",
"\n",
"![](https://huggingface.co/blog/assets/02_how-to-generate/greedy_search.png)\n",
"\n",
"Для каждого слова следующим считали _наиболее вероятное_.\n",
"Но, возможно, пройдя по не самому вероятному слову, в итоге мы получим более вероятную __последовательность__ слов, чем при жадном алгоритме.\n",
"Такой подход называется Beam search.\n",
"\n",
"![](https://huggingface.co/blog/assets/02_how-to-generate/beam_search.png)\n",
"\n",
"[подробнее тут](https://huggingface.co/blog/how-to-generate)\n",
"\n",
"Для использования Beam search передадим в функцию генерации параметр `num_beams` который характеризует количество рассматриваемых \"альтернативных\" путей"
]
},
{
"cell_type": "code",
"execution_count": 105,
"metadata": {
"id": "f5qj2KVoG56d"
},
"outputs": [],
"source": [
"SEED_PHRASE = 'Теория автоматического управления - лучший предмет,'\n",
"input_ids = tokenizer.encode(SEED_PHRASE, return_tensors=\"pt\").to(device)\n",
"model.eval()\n",
"with torch.no_grad():\n",
" out = model.generate(\n",
" input_ids,\n",
" do_sample=False,\n",
" max_new_tokens=75,\n",
" no_repeat_ngram_size=3, # устанавливает вероятность 0 для повторяющихся n-gram (таким образом решается проблема зацикливания)\n",
" pad_token_id=512,\n",
" num_beams=5,\n",
" num_return_sequences=5, # количество возвращенных сгенерированных текстов отранжированных по вероятности после beam_search\n",
" top_p=0.9, #<your choice here>\n",
" top_k=20, #<your choice here>\n",
" temperature=2.0 #<your choice here>\n",
" )\n",
"\n",
"generated_text = list(map(tokenizer.decode, out))\n"
]
},
{
"cell_type": "code",
"execution_count": 106,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "af8foFo1f6Hj",
"outputId": "0f5328b9-201b-486a-b569-30ce7715278a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Applicant 0\n",
"Теория автоматического управления - лучший предмет,\n",
"Который я когда-либо читал.\n",
"\n",
"Я знаю, что это не так,\n",
"Но я не знаю, как это объяснить.\n",
"Я не знаю даже, как объяснить\n",
"То, что я знаю, и то, что не знаю.\n",
"И я знаю только то,\n",
"Что знаю я, но не знаю\n",
"И того, и другого.\n",
"\n",
"\n",
"\n",
"Applicant 1\n",
"Теория автоматического управления - лучший предмет,\n",
"Который я когда-либо читал.\n",
"\n",
"Я знаю, что это не так,\n",
"Но я не знаю, как это объяснить.\n",
"Я не знаю даже, как объяснить\n",
"То, что я знаю, и то, что не знаю.\n",
"И я знаю только то,\n",
"Что знаю я, но не знаю\n",
"И того, что знаю я.\n",
"\n",
"\n",
"Applicant 2\n",
"Теория автоматического управления - лучший предмет,\n",
"Который я когда-либо читал.\n",
"\n",
"Я знаю, что это не так,\n",
"Но я не знаю, как это объяснить.\n",
"Я не знаю даже, как объяснить\n",
"То, что я знаю, и то, что не знаю.\n",
"И я знаю только то,\n",
"Что знаю я, но не знаю\n",
"И то, и другое.\n",
"\n",
"\n",
"\n",
"Applicant 3\n",
"Теория автоматического управления - лучший предмет,\n",
"Который я когда-либо читал.\n",
"\n",
"Я знаю, что это не так,\n",
"Но я не знаю, как это объяснить.\n",
"Я не знаю даже, как объяснить\n",
"То, что я знаю, и то, что не знаю.\n",
"И я знаю только то,\n",
"Что знаю я, но не знаю\n",
"И того, чего не знаю я\n",
"\n",
"\n",
"Applicant 4\n",
"Теория автоматического управления - лучший предмет,\n",
"Который я когда-либо читал.\n",
"\n",
"Я знаю, что это не так,\n",
"Но я не знаю, как это объяснить.\n",
"Я не знаю даже, как объяснить\n",
"То, что я знаю, и то, что не знаю.\n",
"И я знаю только то,\n",
"Что знаю я, но не знаю\n",
"И то, чего не знаю я\n",
"\n",
"\n"
]
}
],
"source": [
"# Выведем _num_return_sequences_ сгенерированных текстов\n",
"for i, seq in enumerate(generated_text):\n",
" print(f\"Applicant {i}\", seq, \"\\n\", sep=\"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AWJBlaMSHzU0"
},
"source": [
" Дополнительно можно изучить смысл параметров top_p и top_k [по ссылке](https://huggingface.co/blog/how-to-generate)\n",
"\n",
"Опробуйте разные значения параметров, какая схема выборки получилась лучше?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NgxotZZrHBLI"
},
"source": [
"Сохранить модель при необходимости:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H8CjiUvzHBn2"
},
"outputs": [],
"source": [
"#torch.save(model, \"gpt2_finetune.torch\")\n",
"#mod = torch.load(\"gpt2_finetune.torch\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8Cn991FqHzU3"
},
"source": [
"### Вопрос 12\n",
"Вместо вывода добавьте лучший сгенерированый текст за лабораторную работу и напишите при какой архитектуре и при каких параметрах он получен:\n",
"\n",
"Например: tuned gpt3 with params:\n",
"\n",
"* do_sample=True,\n",
"* max_new_tokens=80,\n",
"* no_repeat_ngram_size=3,\n",
"* ...\n",
"\n",
"\n",
"В мире, где порядок стремится к хаосу,\n",
"\n",
"Теория управления - свет во тьме.\n",
"\n",
"Автоматы, системы, в них неспроста\n",
"\n",
"Скрыт закон, управляющий временем.\n",
"\n",
"\n",
"Регуляторы, обратные связи,\n",
"\n",
"В этом мире - как волшебный ключ.\n",
"\n",
"С их помощью мы можем, без отказа,\n",
"\n",
"Двигать системы, чьё движенье - дым.\n",
"\n",
"\n",
"\n",
"В каждом узле, в каждом переходе\n",
"\n",
"Скрыта струна, что звучит на удивление чисто.\n",
"\n",
"ТАУ нас учит, как в море перемен,\n",
"\n",
"Найти путь, где устойчивость - не миф, а быль."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Lmhlmlk5lTD3"
},
"source": [
"# Контрольные вопросы\n",
"1. В чем особенность рекуррентных нейронных сетей?\n",
"1. Типы рекуррентных сетей - обычная RNN\n",
"1. Типы рекуррентных сетей - LSTM\n",
"1. Типы рекуррентных сетей - GRU\n",
"1. Что такое и как вычисляется перплексия\n",
"1. Что такое \"предобученная\" модель и для чего ее нужно \"дообучать\"?\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ia7mf_pIlpKe"
},
"outputs": [],
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Ссылка в новой задаче View Git Blame Копировать постоянную ссылку