{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gz18QPRz03Ec"
      },
      "source": [
        "### 1) В среде Google Colab создали новый блокнот (notebook). Импортировали необходимые для работы библиотеки и модули. Настроили блокнот для работы с аппаратным ускорителем GPU."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "mr9IszuQ1ANG"
      },
      "outputs": [],
      "source": [
        "# импорт модулей\n",
        "import os\n",
        "\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.models import Sequential\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FFRtE0TN1AiA"
      },
      "source": [
        "### 2) Загрузили набор данных IMDb, содержащий оцифрованные отзывы на фильмы, размеченные на два класса: позитивные и негативные. При загрузке набора данных параметр seed выбрали равным значению (4k – 1)=31, где k=8 – номер бригады. Вывели размеры полученных обучающих и тестовых массивов данных."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "Ixw5Sp0_1A-w"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[1m17464789/17464789\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m7s\u001b[0m 0us/step\n",
            "Shape of X train: (25000,)\n",
            "Shape of y train: (25000,)\n",
            "Shape of X test: (25000,)\n",
            "Shape of y test: (25000,)\n"
          ]
        }
      ],
      "source": [
        "# загрузка датасета\n",
        "from keras.datasets import imdb\n",
        "\n",
        "vocabulary_size = 5000\n",
        "index_from = 3\n",
        "\n",
        "(X_train, y_train), (X_test, y_test) = imdb.load_data(\n",
        "    path=\"imdb.npz\",\n",
        "    num_words=vocabulary_size,\n",
        "    skip_top=0,\n",
        "    maxlen=None,\n",
        "    seed=31,\n",
        "    start_char=1,\n",
        "    oov_char=2,\n",
        "    index_from=index_from\n",
        "    )\n",
        "\n",
        "# вывод размерностей\n",
        "print('Shape of X train:', X_train.shape)\n",
        "print('Shape of y train:', y_train.shape)\n",
        "print('Shape of X test:', X_test.shape)\n",
        "print('Shape of y test:', y_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aCo_lUXl1BPV"
      },
      "source": [
        "### 3) Вывели один отзыв из обучающего множества в виде списка индексов слов. Преобразовали список индексов в текст и вывели отзыв в виде текста. Вывели длину отзыва. Вывели метку класса данного отзыва и название класса (1 – Positive, 0 – Negative)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "9W3RklPcZyH0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
            "\u001b[1m1641221/1641221\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 2us/step\n"
          ]
        }
      ],
      "source": [
        "# создание словаря для перевода индексов в слова\n",
        "# заргузка словаря \"слово:индекс\"\n",
        "word_to_id = imdb.get_word_index()\n",
        "# уточнение словаря\n",
        "word_to_id = {key:(value + index_from) for key,value in word_to_id.items()}\n",
        "word_to_id[\"<PAD>\"] = 0\n",
        "word_to_id[\"<START>\"] = 1\n",
        "word_to_id[\"<UNK>\"] = 2\n",
        "word_to_id[\"<UNUSED>\"] = 3\n",
        "# создание обратного словаря \"индекс:слово\"\n",
        "id_to_word = {value:key for key,value in word_to_id.items()}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "Nu-Bs1jnaYhB"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[1, 13, 805, 8, 40, 14, 1179, 40, 13, 353, 8, 358, 32, 1179, 108, 13, 384, 3091, 2, 1849, 19, 6, 117, 1006, 5, 49, 836, 89, 70, 25, 140, 355, 21, 2, 13, 104, 9, 35, 463, 7, 15, 2063, 170, 355, 4, 293, 1834, 9, 4, 527, 116, 7, 4, 293, 289, 539, 15, 2, 56, 11, 4, 313, 12, 16, 17, 48, 36, 71, 467, 2, 5, 12, 2230, 72, 39, 126, 397, 928, 11, 68, 4598, 4, 22, 2, 18, 836, 5, 2, 21, 4, 34, 4, 1396, 458, 2, 12, 7, 148, 5, 889, 4, 20, 184, 753, 45, 6, 902, 88, 48, 4, 20, 16, 128, 2142, 12, 62, 28, 28, 77, 2, 4, 65, 5, 105, 26, 184, 948, 5, 50, 26, 49, 465, 5, 2, 1984, 388, 7, 4347, 200, 4, 452, 4, 539, 5, 4, 577, 11, 4, 154, 313, 225, 49, 52, 1006, 5, 2552, 2, 2, 43, 24, 195, 8, 202, 4, 22, 4, 1968, 12, 887, 4, 1962, 9, 184, 2509, 5, 2, 5, 127, 202, 4, 22, 6, 194, 2, 21, 1038, 94, 99, 117, 99, 522, 38, 11, 61, 652, 31, 8, 798, 894, 25, 66, 119, 3720, 1179, 108, 225, 6, 1257, 1166, 7, 986, 21, 4, 22, 1545, 99, 117, 8, 30, 2640]\n",
            "len: 220\n"
          ]
        }
      ],
      "source": [
        "print(X_train[26])\n",
        "print('len:',len(X_train[26]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "JhTwTurtZ6Sp"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<START> i tried to like this slasher like i try to enjoy all slasher films i mean mindless <UNK> mixed with a little nudity and some suspense how can you go wrong but <UNK> i think is an example of that formula going wrong the main issue is the horrible acting of the main three girls that <UNK> up in the house it was as if they were under <UNK> and it stopped me from ever getting interested in their plight the film <UNK> for suspense and <UNK> but the by the numbers direction <UNK> it of those and leaves the movie pretty dull it's a shame because if the movie was better executed it would have have been <UNK> the story and characters are pretty creepy and there are some dark and <UNK> humorous moments of interaction between the mother the girls and the daughter in the old house there's some good nudity and occasional <UNK> <UNK> just not enough to give the film the kick it needed the finale is pretty twisted and <UNK> and does give the film a big <UNK> but sadly its too little too late so in my opinion one to avoid unless you really love obscure slasher films there's a fair amount of potential but the film delivers too little to be worthwhile\n",
            "len: 1159\n",
            "Label: 0 ( Negative )\n"
          ]
        }
      ],
      "source": [
        "review_as_text = ' '.join(id_to_word[id] for id in X_train[26])\n",
        "print(review_as_text)\n",
        "print('len:',len(review_as_text))\n",
        "print('Label:', y_train[26], '(', 'Positive' if y_train[26] == 1 else 'Negative', ')')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4hclnNaD1BuB"
      },
      "source": [
        "### 4) Вывели максимальную и минимальную длину отзыва в обучающем множестве."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "xJH87ISq1B9h"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "MAX Len:  2494\n",
            "MIN Len:  11\n"
          ]
        }
      ],
      "source": [
        "print('MAX Len: ',len(max(X_train, key=len)))\n",
        "print('MIN Len: ',len(min(X_train, key=len)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7x99O8ig1CLh"
      },
      "source": [
        "### 5) Провели предобработку данных. Выбрали единую длину, к которой будут приведены все отзывы. Короткие отзывы дополнили спецсимволами, а длинные обрезали до выбранной длины."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "lrF-B2aScR4t"
      },
      "outputs": [],
      "source": [
        "# предобработка данных\n",
        "from tensorflow.keras.utils import pad_sequences\n",
        "max_words = 500\n",
        "X_train = pad_sequences(X_train, maxlen=max_words, value=0, padding='pre', truncating='post')\n",
        "X_test = pad_sequences(X_test, maxlen=max_words, value=0, padding='pre', truncating='post')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HL2_LVga1C3l"
      },
      "source": [
        "### 6) Повторили пункт 4."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "81Cgq8dn9uL6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "MAX Len:  500\n",
            "MIN Len:  500\n"
          ]
        }
      ],
      "source": [
        "print('MAX Len: ',len(max(X_train, key=len)))\n",
        "print('MIN Len: ',len(min(X_train, key=len)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KzrVY1SR1DZh"
      },
      "source": [
        "### 7) Повторили пункт 3. Сделали вывод о том, как отзыв преобразовался после предобработки."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "vudlgqoCbjU1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[   0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "    1   13  805    8   40   14 1179   40   13  353    8  358   32 1179\n",
            "  108   13  384 3091    2 1849   19    6  117 1006    5   49  836   89\n",
            "   70   25  140  355   21    2   13  104    9   35  463    7   15 2063\n",
            "  170  355    4  293 1834    9    4  527  116    7    4  293  289  539\n",
            "   15    2   56   11    4  313   12   16   17   48   36   71  467    2\n",
            "    5   12 2230   72   39  126  397  928   11   68 4598    4   22    2\n",
            "   18  836    5    2   21    4   34    4 1396  458    2   12    7  148\n",
            "    5  889    4   20  184  753   45    6  902   88   48    4   20   16\n",
            "  128 2142   12   62   28   28   77    2    4   65    5  105   26  184\n",
            "  948    5   50   26   49  465    5    2 1984  388    7 4347  200    4\n",
            "  452    4  539    5    4  577   11    4  154  313  225   49   52 1006\n",
            "    5 2552    2    2   43   24  195    8  202    4   22    4 1968   12\n",
            "  887    4 1962    9  184 2509    5    2    5  127  202    4   22    6\n",
            "  194    2   21 1038   94   99  117   99  522   38   11   61  652   31\n",
            "    8  798  894   25   66  119 3720 1179  108  225    6 1257 1166    7\n",
            "  986   21    4   22 1545   99  117    8   30 2640]\n",
            "len: 500\n"
          ]
        }
      ],
      "source": [
        "print(X_train[26])\n",
        "print('len:',len(X_train[26]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "dbfkWjDI1Dp7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <START> i tried to like this slasher like i try to enjoy all slasher films i mean mindless <UNK> mixed with a little nudity and some suspense how can you go wrong but <UNK> i think is an example of that formula going wrong the main issue is the horrible acting of the main three girls that <UNK> up in the house it was as if they were under <UNK> and it stopped me from ever getting interested in their plight the film <UNK> for suspense and <UNK> but the by the numbers direction <UNK> it of those and leaves the movie pretty dull it's a shame because if the movie was better executed it would have have been <UNK> the story and characters are pretty creepy and there are some dark and <UNK> humorous moments of interaction between the mother the girls and the daughter in the old house there's some good nudity and occasional <UNK> <UNK> just not enough to give the film the kick it needed the finale is pretty twisted and <UNK> and does give the film a big <UNK> but sadly its too little too late so in my opinion one to avoid unless you really love obscure slasher films there's a fair amount of potential but the film delivers too little to be worthwhile\n",
            "len: 2839\n"
          ]
        }
      ],
      "source": [
        "review_as_text = ' '.join(id_to_word[id] for id in X_train[26])\n",
        "print(review_as_text)\n",
        "print('len:',len(review_as_text))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJNRXo5TdPAE"
      },
      "source": [
        "#### После обработки в начало отзыва добавилось необходимое количество токенов <PAD>, чтобы отзыв был длинной в 500 индексов."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YgiVGr5_1D3u"
      },
      "source": [
        "### 8) Вывели предобработанные массивы обучающих и тестовых данных и их размерности."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "7MqcG_wl1EHI"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "X train: \n",
            " [[   0    0    0 ...    2 4050    2]\n",
            " [   0    0    0 ...  721   90  180]\n",
            " [   0    0    0 ... 1114    2  174]\n",
            " ...\n",
            " [   1 1065 2022 ...    7 1514    2]\n",
            " [   0    0    0 ...    6  879  132]\n",
            " [   0    0    0 ...   12  152  157]]\n",
            "X train: \n",
            " [[   0    0    0 ...   10  342  158]\n",
            " [   0    0    0 ...    2   67   12]\n",
            " [   0    0    0 ... 1242 1095 1095]\n",
            " ...\n",
            " [   0    0    0 ...    4    2  136]\n",
            " [   0    0    0 ...   14   31  591]\n",
            " [   0    0    0 ...    7 3923  212]]\n",
            "Shape of X train: (25000, 500)\n",
            "Shape of X test: (25000, 500)\n"
          ]
        }
      ],
      "source": [
        "# вывод данных\n",
        "print('X train: \\n',X_train)\n",
        "print('X train: \\n',X_test)\n",
        "\n",
        "# вывод размерностей\n",
        "print('Shape of X train:', X_train.shape)\n",
        "print('Shape of X test:', X_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "amaspXGW1EVy"
      },
      "source": [
        "### 9) Реализовали модель рекуррентной нейронной сети, состоящей из слоев Embedding, LSTM, Dropout, Dense, и обучили ее на обучающих данных с выделением части обучающих данных в качестве валидационных. Вывели информацию об архитектуре нейронной сети. Добились качества обучения по метрике accuracy не менее 0.8."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "ktWEeqWd1EyF"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "c:\\Users\\Admin\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\keras\\src\\layers\\core\\embedding.py:97: UserWarning: Argument `input_length` is deprecated. Just remove it.\n",
            "  warnings.warn(\n",
            "c:\\Users\\Admin\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\keras\\src\\layers\\core\\embedding.py:100: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n",
            "  super().__init__(**kwargs)\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"sequential\"</span>\n",
              "</pre>\n"
            ],
            "text/plain": [
              "\u001b[1mModel: \"sequential\"\u001b[0m\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
              "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
              "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">500</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">160,000</span> │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ dropout (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dropout</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ dense (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                   │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1</span>)              │            <span style=\"color: #00af00; text-decoration-color: #00af00\">65</span> │\n",
              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
              "</pre>\n"
            ],
            "text/plain": [
              "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
              "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
              "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m500\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m160,000\u001b[0m │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ dropout (\u001b[38;5;33mDropout\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
              "│ dense (\u001b[38;5;33mDense\u001b[0m)                   │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1\u001b[0m)              │            \u001b[38;5;34m65\u001b[0m │\n",
              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">184,897</span> (722.25 KB)\n",
              "</pre>\n"
            ],
            "text/plain": [
              "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m184,897\u001b[0m (722.25 KB)\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">184,897</span> (722.25 KB)\n",
              "</pre>\n"
            ],
            "text/plain": [
              "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m184,897\u001b[0m (722.25 KB)\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
              "</pre>\n"
            ],
            "text/plain": [
              "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "embed_dim = 32\n",
        "lstm_units = 64\n",
        "\n",
        "model = Sequential()\n",
        "model.add(layers.Embedding(input_dim=vocabulary_size, output_dim=embed_dim, input_length=max_words, input_shape=(max_words,)))\n",
        "model.add(layers.LSTM(lstm_units))\n",
        "model.add(layers.Dropout(0.5))\n",
        "model.add(layers.Dense(1, activation='sigmoid'))\n",
        "\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "CuPqKpX0kQfP"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m61s\u001b[0m 184ms/step - accuracy: 0.8464 - loss: 0.3649 - val_accuracy: 0.8366 - val_loss: 0.3726\n",
            "Epoch 2/5\n",
            "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m58s\u001b[0m 184ms/step - accuracy: 0.8838 - loss: 0.2931 - val_accuracy: 0.8692 - val_loss: 0.3221\n",
            "Epoch 3/5\n",
            "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m59s\u001b[0m 188ms/step - accuracy: 0.9015 - loss: 0.2519 - val_accuracy: 0.8652 - val_loss: 0.3294\n",
            "Epoch 4/5\n",
            "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m58s\u001b[0m 185ms/step - accuracy: 0.9151 - loss: 0.2225 - val_accuracy: 0.8636 - val_loss: 0.3255\n",
            "Epoch 5/5\n",
            "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m82s\u001b[0m 184ms/step - accuracy: 0.9162 - loss: 0.2174 - val_accuracy: 0.8660 - val_loss: 0.3360\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<keras.src.callbacks.history.History at 0x219fd150250>"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# компилируем и обучаем модель\n",
        "batch_size = 64\n",
        "epochs = 5\n",
        "model.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "hJIWinxymQjb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m38s\u001b[0m 49ms/step - accuracy: 0.8659 - loss: 0.3349\n",
            "\n",
            "Test accuracy: 0.865880012512207\n"
          ]
        }
      ],
      "source": [
        "test_loss, test_acc = model.evaluate(X_test, y_test)\n",
        "print(f\"\\nTest accuracy: {test_acc}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mgrihPd61E8w"
      },
      "source": [
        "### 10) Оценили качество обучения на тестовых данных:\n",
        "### - вывели значение метрики качества классификации на тестовых данных\n",
        "### - вывели отчет о качестве классификации тестовой выборки  \n",
        "### - построили ROC-кривую по результату обработки тестовой выборки и вычислили площадь под ROC-кривой (AUC ROC)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "Rya5ABT8msha"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Test accuracy: 0.865880012512207\n"
          ]
        }
      ],
      "source": [
        "#значение метрики качества классификации на тестовых данных\n",
        "print(f\"\\nTest accuracy: {test_acc}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "2kHjcmnCmv0Y"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m40s\u001b[0m 50ms/step\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "    Negative       0.91      0.82      0.86     12500\n",
            "    Positive       0.83      0.92      0.87     12500\n",
            "\n",
            "    accuracy                           0.87     25000\n",
            "   macro avg       0.87      0.87      0.87     25000\n",
            "weighted avg       0.87      0.87      0.87     25000\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#отчет о качестве классификации тестовой выборки\n",
        "y_score = model.predict(X_test)\n",
        "y_pred = [1 if y_score[i,0]>=0.5 else 0 for i in range(len(y_score))]\n",
        "\n",
        "from sklearn.metrics import classification_report\n",
        "print(classification_report(y_test, y_pred, labels = [0, 1], target_names=['Negative', 'Positive']))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Kp4AQRbcmwAx"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABRcElEQVR4nO3deXgT5doG8DtJk7TpXkpXimVfZAfhsIlgWURRXACFDyoqHoUqh4qyCFRUqIoiHkURECuIgrscQaCgVTYFgSIIFKGUCrSFUmi6Jmnyfn+kDcS22JQk0yT377q4mkxmJk8eKrmdeWdemRBCgIiIiMhNyKUugIiIiMieGG6IiIjIrTDcEBERkVthuCEiIiK3wnBDREREboXhhoiIiNwKww0RERG5FYYbIiIicisMN0RERORWGG6IiIjIrTDcEFGDkpKSAplMZvnj5eWF6OhoPPzwwzh37ly19YUQWLNmDW699VYEBQVBo9GgY8eOePHFF1FSUlLr+3z99de44447EBoaCpVKhaioKIwePRo//PCDIz8eETmBjHNLEVFDkpKSgokTJ+LFF19Es2bNUF5ejl9++QUpKSmIjY3FkSNH4O3tDQAwGo0YO3YsPvvsM/Tv3x/33XcfNBoNduzYgU8++QTt27fHtm3bEB4ebtm/EAKPPPIIUlJS0LVrVzzwwAOIiIhATk4Ovv76a+zfvx+7du1Cnz59pGoBEd0oQUTUgHz44YcCgNi3b5/V8hkzZggAYv369ZZlCxcuFADE9OnTq+1nw4YNQi6Xi2HDhlktX7RokQAg/vOf/wiTyVRtu9WrV4tff/3VTp+GiKTA01JE5BL69+8PADh16hQAoKysDIsWLULr1q2RnJxcbf0RI0YgPj4emzdvxi+//GLZJjk5GW3btsXrr78OmUxWbbvx48ejZ8+eDvwkRORoDDdE5BKysrIAAMHBwQCAnTt34vLlyxg7diy8vLxq3GbChAkAgO+++86yTUFBAcaOHQuFQuH4oolIEjX/i0BEJLHCwkLk5+ejvLwcv/76K+bPnw+1Wo277roLAHD06FEAQOfOnWvdR9Vrx44ds/rZsWNHR5ZORBJjuCGiBikuLs7qeWxsLD7++GM0adIEAFBUVAQA8Pf3r3UfVa9ptVqrn9fbhohcH8MNETVIS5cuRevWrVFYWIhVq1bh559/hlqttrxeFVCqQk5N/h6AAgIC/nEbInJ9HHNDRA1Sz549ERcXh/vvvx8bNmxAhw4dMHbsWBQXFwMA2rVrBwD4/fffa91H1Wvt27cHALRt2xYAcPjwYUeWTkQSY7ghogZPoVAgOTkZ58+fxzvvvAMA6NevH4KCgvDJJ5/AaDTWuN3q1asBwDJOp1+/fggODsann35a6zZE5PoYbojIJdx2223o2bMnlixZgvLycmg0GkyfPh0ZGRl4/vnnq62/ceNGpKSkYOjQofjXv/4FANBoNJgxYwaOHTuGGTNmQNRwD9OPP/4Ye/fudfjnISLH4ZgbInIZzz77LEaNGoWUlBQ88cQTmDlzJg4ePIhXX30Ve/bswf333w8fHx/s3LkTH3/8Mdq1a4ePPvqo2j7++OMPvPHGG/jxxx8tdyjOzc3FN998g71792L37t0SfUIisgdOv0BEDUrV9Av79u1Djx49rF4zmUxo3bo1ACAjIwMKhQImkwmrV6/GypUrcfjwYej1erRo0QKjR4/GM888A19f3xrf58svv8Ty5cvx22+/QavVonHjxrj11lvx5JNPYsCAAQ7/nETkOAw3RERE5FY45oaIiIjcCsMNERERuRWGGyIiInIrDDdERETkVhhuiIiIyK0w3BAREZFb8bib+JlMJpw/fx7+/v6QyWRSl0NERER1IIRAUVERoqKiIJdf/9iMx4Wb8+fPIyYmRuoyiIiIqB7++usvNGnS5LrreFy48ff3B2BuTkBAgF33bTAYsHXrVgwZMgRKpdKu+6ar2GfnYJ+dg312HvbaORzVZ61Wi5iYGMv3+PV4XLipOhUVEBDgkHCj0WgQEBDA/3AciH12DvbZOdhn52GvncPRfa7LkBIOKCYiIiK3wnBDREREboXhhoiIiNwKww0RERG5FYYbIiIicisMN0RERORWGG6IiIjIrTDcEBERkVthuCEiIiK3wnBDREREbkXScPPzzz9jxIgRiIqKgkwmwzfffPOP26SlpaFbt25Qq9Vo2bIlUlJSHF4nERERuQ5Jw01JSQk6d+6MpUuX1mn906dP484778TAgQORnp6O//znP3jsscewZcsWB1dKRERErkLSiTPvuOMO3HHHHXVef9myZWjWrBneeOMNAEC7du2wc+dOvPnmmxg6dKijyiQiInIqIUTlT0Bcs+zqY6DqmRDXbme9vLZtq54ICMv24tr3vWa/AgIVRgHTtW90HRUVFdDq6/pJHcOlZgXfs2cP4uLirJYNHToU//nPf2rdRqfTQafTWZ5rtVoA5llLDQaDXeur2p+990vW2GfnYJ+dwxX7LIRAucEEXYUJFSYTDEaBCpMJxeVGCAiYTIBJiMo/1zy2LAdK9RUoN5igVMhQYRIwmQQqTAJGk4BRCJy5VIpGfiqYTOb3q9qPEFf3If62/7OXy6BRKeCtVMAkAKBqvcp1TCacOyfHts8OATKZ+cu/MgxU7U8AKNUbkZVfgthQX6v3/PvPv9cgBFBQqseFIh1ign1gMplfM16zjdFkfmwU5s9cojdCLrMOE+4g1k+B+x30HVsXLhVucnNzER4ebrUsPDwcWq0WZWVl8PHxqbZNcnIy5s+fX2351q1bodFoHFJnamqqQ/ZL1thn52CfnSM1NdX8BQjAaAL0JkBnBEoqgAoTYBQyVJiASzrASw4YhXm9qvVzy2QIUsH8hYvaf5ZXAAU6GYJUAhXC/D5nimSI0KDyS9q8T3HtdgKoEMAVvQzeCoFyo0y6Rt0wOZCfV6c1z14pr/e7ZBeU1XldkwuFGqWsbsV6ye3/b0dpaWmd13WpcFMfs2bNQmJiouW5VqtFTEwMhgwZgoCAALu+l8FgQGpqKgYPHgylUmnXfdNV7LNzeHKfhRAwGAX0RhO0ZQaU6o0wGAUMRhMKSvUwCeBKqR75xXqoveSoMAkYKkwo1RuRfbkMIb4q6AxGHPzrCpqGaKCvMMFgNGHfmStoEep7zdEOgZzCcqjkAiaY9+M81gHlTHHdtqop2CgVMnjJZZDJZCjVGxEeoIZCJoNcBsjlMshlVX9g/ik3P84pLEeIrwph/mrIZeZ9KCr/yGVAZn4JOkYHWraVXbuPvz2XyQCFXIaLxXqEaJTwU3tZlstl5pqFMOHknyfQpnVreHl5QSYzd6FqPRnM+5TJAKNJQF9hQqif2uq9/v6e19ZS9bzCaIK3UgGVl7yyDzLI5ebXFNfUWrUPpUJe2b9r/mYqn8iuPoQM1uvILH8VlevKrv6tmh9XXw6Z7G/rVD2u4f2uvoFNHPVvR9WZl7pwqXATERGBvDzrxJ2Xl4eAgIAaj9oAgFqthlqtrrZcqVQ67B9sR+6brmKfncMV+6yrMEJbVoFLJTqU6Y04d6UMJgGUG4zQVZgDS4nOfFokI08LpUKOy6UG5BfpcO5KGWQy+50i+PNCidXzExeqpwi9SYaroyOsBWuUuFxqQKswP6i85FB5yXEyrxi9WzSCUiGHQi6DV2W4yLpUipujAsxfqJVfngo5rgkX5ucVlV/cMSEayxerwWj+Iq/64q3aR1XQMG8rg9pLgUAfJTRqBTRKBRSVocYVGAwGbCrJwPBbW7jc77Qrsve/Hbbsy6XCTe/evbFp0yarZampqejdu7dEFRGRIwhhPmpSWGpATmE5SnQVyCsqx8UiHXILdVB6yXAyrxhKhRwyGXA0RwsfpQLHc4vs9P7Vl4X5q6FUmMPF6fwS9GoWAoPRBKMAmof6wksug9JLDlE5nqRlYz+olXIUlVcgtpEvlF5yqBRyGIwmhPmr4aUwP4cwYs+uHRgWNwjeaiWUcjm8FOYQofLirciI6kPScFNcXIyTJ09anp8+fRrp6ekICQlB06ZNMWvWLJw7dw6rV68GADzxxBN455138Nxzz+GRRx7BDz/8gM8++wwbN26U6iMQ0T/QV5iQX6zDkXOFKNFXIKewHIVlBsggw4WichSU6LHn1CU0DdHgzwvF0KgUKDcY7ToOoVWYH05dLMatrRvD20sBH5X56IO3UgG1l/noR6swP4T4qhDiq4Kftxc0Ki+oKwOJXO64IxMGgwGZPkBkoDePJhDZiaTh5rfffsPAgQMtz6vGxsTHxyMlJQU5OTnIzs62vN6sWTNs3LgR06ZNw1tvvYUmTZpg5cqVvAycyMmKyg3Iyi/FX5dLkZFbhFMXi5FdUIpiXQXUXgocy9HCV6VAid5Y533+WXm6pvRv2/irvdDIT4VAjQqlugrzYx8lWoX5o0RfgahAHwRqlNBVmNA0RINGvipEBfkgyEfp0FBCRA2XpOHmtttus1xTX5Oa7j5822234eDBgw6sishzFJYakKMtQ5neiCulBmjLDSjTG1FYZkB+UTlSDyvwbuZuGEwCXnIZTuTVcdQpUC3YRAV6o6i8Ar2ah0AIwEelQIvGfogI9Ia/txe85DJEBfnAR2k+qqJWKuCv9mJAISKbudSYGyL6Z4bKK3zyi/XIKSzDuStluFyixx/ntfBTe0FbbsDWo3l1HDArA4prDzSdmgQiIsAbaqUCXnIZmoX6onW4Pxr5qaBSyBHiq4JGpUCwRsWQQkROw3BD5GKEMF8+XFCix5lLpfjpxAWcvFCMS5XP60Mhl6FlYz8EVV5C661SIMTHC3lnz6B5ixZo3tgfYQFqBPgo4a/2qjzawvEhRNQwMdwQNUBCCFwuNSCnsAxnLpUi/a8ruKAtR3ZBKf68UIyi8op/3IeX3HznV5VCjs4xgQjxVcFLLkev5iEI81ejSbAGzRv7QqOq+Z8Bg8GATZtOY/jgVhzoSkQuheGGyMn0FSb8eaEIf+YVo1RvxOFzhSjRVSC/WIcTecXIL9b9804ANPJVITrYx3wZskKOZqG+aNHYFy3D/NAs1A8KngYiIg/FcEPkQAajCfvPXMaX+8/i8LlCXCjSoaDE9hnlwgPU6N28EQa2DUOTYB/cHBUIb6XCARUTEbk+hhuiG2Q0CfOl0ZdK8fWBszhfWI7fsgrgq/bC2cvXn18mIsAb3WODUa43on1UAII15suYmzf2RdMQDQMMEVE9MNwQ2aBUX4Gj57U4c6kUqUfzcPhcIS6V6FBuMFVb93Kp9Qy2UYHemDKoJXo1C0GLxn4uc8t6IiJXw3BDVAshBHadvIRTF4vx0e4slOqNyNVef5bgpiEaNAv1xb+aN0Lzxr7o1CQQ4f7evAyaiMiJGG6IrlGmN+Jg9mV8dfAc0jIu1jq495bYYDQL9UXHJkFoF+GPTk2COA8QEVEDwXBDHu1yiR5bj+biYPYVfJN+rtrpJZVCDrWXHPd1i0bTRr74V/MQtI0I4JVIREQNGMMNeaTsS6V47stD+CWzoMbX/dReeKhnDBIGtkKghvd4ISJyJQw35DFMJoEfjl/Aq5uPWyZpBMz3i+nV3DzI9/5uTXBTIw0H+xIRuTCGG3J7+goTvj+Sg3d/PIWMvCLL8s5NAvHs0Lbo27IRwwwRkRthuCG3lF+sw4odmdifdRm/nbls9dp93aLx5IAWaBXuL1F1RETkSAw35DZ2nczH7lP5WLf3L1yq4S7A7SIDsDK+B6KDfCSojoiInIXhhlxWsa4CvxzPx9EcLdb+ml3jtAaD2obhkb7NeOqJiMiDMNyQSxFC4MeMi/jPHgXEnh9qXOepQS0xuH04OjUJcm5xRETUIDDckEsQQiBldxbW/pqNkxeKAZiPwqgUcsSGajCkfQTG3BKDmBCNtIUSEZHkGG6owTt7uRSJnx3C3tNX70nTwl/g6Ts64+6uTXi6iYiIrDDcUINkNJmP1Lz03VGr5T1uCsYLd7XFqQM7MLxjBIMNERFVw3BDDUr2pVK8+N0f2HbsQrXXvnyyD7rfFAyDwYBTEtRGRESugeGGJKevMGH1nixsOHQev58ttHrtlthgPNK3Ge7oGClRdURE5GoYbkgyu0/lY+43R3DqYkm116YPaY2JfZvBV81fUSIisg2/OcjpLpfoEf/h3mpHaW5vG4Y7O0Xivm5NJKqMiIjcAcMNOdXB7Mu4993dVsvm330zxtwSA2+lQqKqiIjInTDckFMIIfDxr9mY+80Ry7KxvZpiwcgOvOKJiIjsiuGGHO73s1fw/NdHcPjc1dNQW/5zK9pEcOJKIiKyP4Ybcqh1e7Mx86vDludP3tYCzw1tw6M1RETkMAw35BAXi3R4L+0UVu06DQBQKmT4enJfdIgOlLgyIiJydww3ZFdCCLybdgpLtp2AwSgAANFBPtj+zAAOGCYiIqdguCG7KSo34J6lu5BZed8af28vTL29FeL7xEKpkEtcHREReQqGG7phZy6V4OWNx5B6NM+y7KGeTfHC3e2h9uLRGiIici6GG6o3o0ng2/RzSPzskGWZt1KOl+7pgFE9YiSsjIiIPBnDDdVLWsYFTF2XjsIyg2VZ8n0dcW/XaI6tISIiSTHckE1MJoFXtxzH+z9lWpY90rcZnhjQHGEB3hJWRkREZMZwQ3V28kIRJq89gBN5xQCA1uF++CD+FsSEaCSujIiI6CqGG6qTjNwiPLBsN4rKKwAA4/91E+aNaM+roIiIqMFhuKF/tPtkPsau/BWA+fLuTx77Fzo24c34iIioYWK4oetKy7iAhz/cBwCQy4ANCf3QLNRX4qqIiIhqx3BDtTp5odgSbADgh2duQyyDDRERNXAcMEE1+jHjAuIW/2R5/r+Efgw2RETkEnjkhqr5MeMCJlYesVF5ybHxqX5oFe4vcVVERER1w3BDVr45eA7/WZ9ueZ42/TZEBflIVxAREZGNeFqKLNL/umIJNo18Vdg9cxCDDRERuRweuSEAwK+ZlzDlkwMAAIVchq3TbkUjP7XEVREREdmO4YbwS+YlPLj8FwDmy72/erIPgw0REbkshhsPdyD7siXYKOQy7Jk5iHNEERGRS+OYGw+WlV+C+97dbXmeNv02BhsiInJ5DDceqtxgxG2vp1meLxnThRNgEhGRW2C48VBr9pyxPH7t/k4Y2TVawmqIiIjsh+HGA+kqjFiw6RgAoH+rUIy+JUbiioiIiOyH4cbDCCGQ+Nkhy/PXR3WWsBoiIiL7Y7jxMG9u+xMbf88BADw7tA3COYCYiIjcDMONB9n6Ry7+u/1PAMBDPZtiysCWEldERERkfww3HkIIgcfX7AcAtArzw4KRHSSuiIiIyDEYbjxAYZkBw/+70/J81cO3QC6XSVgRERGR40gebpYuXYrY2Fh4e3ujV69e2Lt373XXX7JkCdq0aQMfHx/ExMRg2rRpKC8vd1K1rkcIgf9b+SuO5WgBALOHt+X9bIiIyK1JGm7Wr1+PxMREJCUl4cCBA+jcuTOGDh2KCxcu1Lj+J598gpkzZyIpKQnHjh3DBx98gPXr12P27NlOrtx1HDpbiMPnCgGYBxA/fmsLiSsiIiJyLEnDzeLFizFp0iRMnDgR7du3x7Jly6DRaLBq1aoa19+9ezf69u2LsWPHIjY2FkOGDMFDDz30j0d7PJXRJDB2hXneqB43BXMAMREReQTJJs7U6/XYv38/Zs2aZVkml8sRFxeHPXv21LhNnz598PHHH2Pv3r3o2bMnMjMzsWnTJowfP77W99HpdNDpdJbnWq359IzBYIDBYLDTp4Fln9f+lNp/fziJUr0RAHBf18gGU9eNamh9dlfss3Owz87DXjuHo/psy/4kCzf5+fkwGo0IDw+3Wh4eHo7jx4/XuM3YsWORn5+Pfv36QQiBiooKPPHEE9c9LZWcnIz58+dXW75161ZoNI4Ze5KamuqQ/dpCbwTe3mv+6+0XboJv3u/YtOl3iauyr4bQZ0/APjsH++w87LVz2LvPpaWldV5XsnBTH2lpaVi4cCHeffdd9OrVCydPnsTUqVPx0ksvYe7cuTVuM2vWLCQmJlqea7VaxMTEYMiQIQgICLBrfQaDAampqRg8eDCUSqVd922rN7edBJAJf28vLPv3AKiVCknrsaeG1Gd3xj47B/vsPOy1cziqz1VnXupCsnATGhoKhUKBvLw8q+V5eXmIiIiocZu5c+di/PjxeOyxxwAAHTt2RElJCR5//HE8//zzkMurDyFSq9VQq9XVliuVSof9cjty33XxW1YB3v0pEwAwvEMk/DTueRdiqfvsKdhn52CfnYe9dg5799mWfUk2oFilUqF79+7Yvn27ZZnJZML27dvRu3fvGrcpLS2tFmAUCvMRCSGE44p1IdpyAx7+cB8AIMRXhaS720tcERERkXNJeloqMTER8fHx6NGjB3r27IklS5agpKQEEydOBABMmDAB0dHRSE5OBgCMGDECixcvRteuXS2npebOnYsRI0ZYQo4nE0Jg5Du7UKyrAGC+WZ9G5VJnHomIiG6YpN98Y8aMwcWLFzFv3jzk5uaiS5cu2Lx5s2WQcXZ2ttWRmjlz5kAmk2HOnDk4d+4cGjdujBEjRmDBggVSfYQGZfWeM8jMLwEAPD+8HbrEBElbEBERkQQk/9/6hIQEJCQk1PhaWlqa1XMvLy8kJSUhKSnJCZW5FiEENh42z/bd2F+NSbc2l7giIiIiaUg+/QLZx8e/ZmPv6QIAwMucFJOIiDwYw40bEEJg7jdHAAAtGvsirl34P2xBRETkvhhu3MCHu7Isj1c9fAsUnPGbiIg8GMONi6swmrBoSwYA4F/NQ3BTI1+JKyIiIpIWw42Le3njMZQZzPNHLby3o8TVEBERSY/hxoVdLNIhZXcWAODhPrFo3thP2oKIiIgaAIYbF3bLgm2WxwmDWkpYCRERUcPBcOOiDmRftjyeFtcaoX7V588iIiLyRAw3LuqL/Wctj6fGtZKwEiIiooaF4cYFGYwmfF95N+I5d7aTuBoiIqKGheHGBb29/U9cLjVAIZfh3q7RUpdDRETUoDDcuKA1v5wBAHS/KRiNONaGiIjICsONi8nKL8HlUgMAIGEgr5AiIiL6O4YbF/P+z5mWx/1bhUpYCRERUcPEcONCCkr0+HRvNgBgxrC2kMk4hxQREdHfMdy4kCc+3m95PLFvrHSFEBERNWAMNy6i3GDE3tMFAIBnBreGt1IhcUVEREQNE8ONi1j20ynL48cHNJewEiIiooaN4cZFVIWb/q1CofbiURsiIqLaMNy4gCXbTqDcYAIAzLurvcTVEBERNWwMNy7g3R/NR22ah/qiVbi/xNUQERE1bAw3DVz6X1egN5qP2vz3oa4SV0NERNTwMdw0cO9fM5C4Q3SghJUQERG5BoabBkxbbsD3R3IBAG+O6SxxNURERK6B4aYBS9mVBQDwVSkwolOUtMUQERG5CIabBmztr+bZv+/uEgUvBf+qiIiI6oLfmA1URm4R8rQ6AMDk2zj7NxERUV0x3DRQ76adtDyOCdFIWAkREZFrYbhpoLLySwAAo3s0kbgSIiIi18Jw0wAVlOhx6GwhAGD8v2KlLYaIiMjFMNw0QEu2nQAAKOQytI7wk7gaIiIi18Jw0wD9fOIiAOD+btGcJJOIiMhGDDcNjMFoQtalUgDAXby3DRERkc1uKNyUl5fbqw6qdPS81vK4b8tQCSshIiJyTTaHG5PJhJdeegnR0dHw8/NDZmYmAGDu3Ln44IMP7F6gp9mXVQAA8FbKoZDLJK6GiIjI9dgcbl5++WWkpKTgtddeg0qlsizv0KEDVq5cadfiPNGHlVMuxPeJlbQOIiIiV2VzuFm9ejWWL1+OcePGQaG4Oti1c+fOOH78uF2L8zTlBiPOXSkDAM4lRUREVE82h5tz586hZcvq0wGYTCYYDAa7FOWpfsu6bHl8c1SAhJUQERG5LpvDTfv27bFjx45qy7/44gt07drVLkV5qs1/5Fgey2Qcb0NERFQfXrZuMG/ePMTHx+PcuXMwmUz46quvkJGRgdWrV+O7775zRI0eY/fJSwCAW1s3lrgSIiIi12XzkZt77rkH//vf/7Bt2zb4+vpi3rx5OHbsGP73v/9h8ODBjqjRY+RpzZfWd2saJG0hRERELszmIzcA0L9/f6Smptq7Fo9WYTShRG8EAAzgkRsiIqJ6s/nITfPmzXHp0qVqy69cuYLmzZvbpShP9OWBs5bH7SI5mJiIiKi+bA43WVlZMBqN1ZbrdDqcO3fOLkV5ov1nrl4p5a3kfFJERET1VefTUhs2bLA83rJlCwIDAy3PjUYjtm/fjtjYWLsW50kKSsyX0d/ZMVLiSoiIiFxbncPNyJEjAZgvUY6Pj7d6TalUIjY2Fm+88YZdi/MUQgj8mmk+1cfxNkRERDemzuHGZDIBAJo1a4Z9+/YhNJSTOtpLQYkeRboKAMBdnXnkhoiI6EbYfLXU6dOnHVGHR6uaLDPEVwWNql4XsBEREVGlen2TlpSU4KeffkJ2djb0er3Va08//bRdCvMke0+bBxMHa5QSV0JEROT6bA43Bw8exPDhw1FaWoqSkhKEhIQgPz8fGo0GYWFhDDf1cPjcFQBAeIC3tIUQERG5AZsvBZ82bRpGjBiBy5cvw8fHB7/88gvOnDmD7t274/XXX3dEjW5vX+WEmXd0iJC4EiIiItdnc7hJT0/HM888A7lcDoVCAZ1Oh5iYGLz22muYPXu2I2p0ayWVA4kBoHcLDtImIiK6UTaHG6VSCbncvFlYWBiys7MBAIGBgfjrr7/sW50H2PFnPgBAJgNaNPaVuBoiIiLXZ/OYm65du2Lfvn1o1aoVBgwYgHnz5iE/Px9r1qxBhw4dHFGjWzv4l/mUVESAN2QymcTVEBERuT6bj9wsXLgQkZHme7EsWLAAwcHBePLJJ3Hx4kW8//77di/Q3RWVm09LtYnwl7gSIiIi92DzkZsePXpYHoeFhWHz5s12LcjTbP0jFwBwe9swiSshIiJyDzYfuanNgQMHcNddd9m83dKlSxEbGwtvb2/06tULe/fuve76V65cwZQpUxAZGQm1Wo3WrVtj06ZN9S1bcvnF5vsERQb6SFwJERGRe7Ap3GzZsgXTp0/H7NmzkZmZCQA4fvw4Ro4ciVtuucUyRUNdrV+/HomJiUhKSsKBAwfQuXNnDB06FBcuXKhxfb1ej8GDByMrKwtffPEFMjIysGLFCkRHR9v0vg3FpWKd5XGnmMDrrElERER1VefTUh988AEmTZqEkJAQXL58GStXrsTixYvx1FNPYcyYMThy5AjatWtn05svXrwYkyZNwsSJEwEAy5Ytw8aNG7Fq1SrMnDmz2vqrVq1CQUEBdu/eDaXSfDdfV56J/M8LxZbHYf68gR8REZE91DncvPXWW3j11Vfx7LPP4ssvv8SoUaPw7rvv4vDhw2jSpInNb6zX67F//37MmjXLskwulyMuLg579uypcZsNGzagd+/emDJlCr799ls0btwYY8eOxYwZM6BQKGrcRqfTQae7eoREq9UCAAwGAwwGg811X0/V/uq63zP5RQAAL7nM7rW4M1v7TPXDPjsH++w87LVzOKrPtuyvzuHm1KlTGDVqFADgvvvug5eXFxYtWlSvYAMA+fn5MBqNCA8Pt1oeHh6O48eP17hNZmYmfvjhB4wbNw6bNm3CyZMnMXnyZBgMBiQlJdW4TXJyMubPn19t+datW6HRaOpV+z9JTU2t03p7zssAKNBIbXLpcUNSqWuf6cawz87BPjsPe+0c9u5zaWlpndetc7gpKyuzhAGZTAa1Wm25JNxZTCYTwsLCsHz5cigUCnTv3h3nzp3DokWLag03s2bNQmJiouW5VqtFTEwMhgwZgoCAALvWZzAYkJqaisGDB1tOm13Pp6v2AbiM225uguHDb7ZrLe7M1j5T/bDPzsE+Ow977RyO6nPVmZe6sOlS8JUrV8LPzw8AUFFRgZSUFISGWk8ZUNeJM0NDQ6FQKJCXl2e1PC8vDxERNc+xFBkZCaVSaXUKql27dsjNzYVer4dKpaq2jVqthlqtrrZcqVQ67Je7rvtWepk/R2SQhv+h1YMj/w7pKvbZOdhn52GvncPefbZlX3UON02bNsWKFSsszyMiIrBmzRqrdWQyWZ3DjUqlQvfu3bF9+3aMHDkSgPnIzPbt25GQkFDjNn379sUnn3wCk8lkmQLixIkTiIyMrDHYNHQ7T5qnXmjLG/gRERHZTZ3DTVZWlt3fPDExEfHx8ejRowd69uyJJUuWoKSkxHL11IQJExAdHY3k5GQAwJNPPol33nkHU6dOxVNPPYU///wTCxcurHOgamiEMP/0U/P/IIiIiOzF5jsU29OYMWNw8eJFzJs3D7m5uejSpQs2b95sGWScnZ1tOUIDADExMdiyZQumTZuGTp06ITo6GlOnTsWMGTOk+gj1dkFbbnncPsq+Y3+IiIg8maThBgASEhJqPQ2VlpZWbVnv3r3xyy+/OLgqx/u5cjZwX5UCIb6ud0qNiIioobLb9AtkmxN55nvctAjzk7gSIiIi98JwI5FfTxcAAOLahf/DmkRERGQLhhuJnKw8cqPy4l8BERGRPdXrm/XUqVOYM2cOHnroIcskl99//z3++OMPuxbnzgJ9KufGauQrcSVERETuxeZw89NPP6Fjx4749ddf8dVXX6G42Dz546FDh2q9SzBVd77QfLVUyzCGGyIiInuyOdzMnDkTL7/8MlJTU61unDdo0CC3uIrJGQrLrk7+FepX/e7JREREVH82h5vDhw/j3nvvrbY8LCwM+fn5dinK3R3PMc+PoVTIEKThZeBERET2ZHO4CQoKQk5OTrXlBw8eRHR0tF2KcndZl0oAAN1vCpa4EiIiIvdjc7h58MEHMWPGDOTm5kImk8FkMmHXrl2YPn06JkyY4Iga3c6Oyhv4NQvlPW6IiIjszeZws3DhQrRt2xYxMTEoLi5G+/btceutt6JPnz6YM2eOI2p0O9/9bj7y1TREI3ElRERE7sfm6RdUKhVWrFiBuXPn4siRIyguLkbXrl3RqlUrR9TndkwmYXkc24jhhoiIyN5sDjc7d+5Ev3790LRpUzRt2tQRNbm1Q2evWB4PbBsmXSFERERuyubTUoMGDUKzZs0we/ZsHD161BE1ubWzl8ssj72VCgkrISIick82h5vz58/jmWeewU8//YQOHTqgS5cuWLRoEc6ePeuI+tzOtmN5AIB+LUMlroSIiMg92RxuQkNDkZCQgF27duHUqVMYNWoUPvroI8TGxmLQoEGOqNGt7KucMNN4zdgbIiIisp8bmrWxWbNmmDlzJl555RV07NgRP/30k73qcltV0y6M7cXxSkRERI5Q73Cza9cuTJ48GZGRkRg7diw6dOiAjRs32rM2tyOEgEIuAwB0iA6UuBoiIiL3ZPPVUrNmzcK6detw/vx5DB48GG+99RbuueceaDS8rPmflOqNltNRoX6cdoGIiMgRbA43P//8M5599lmMHj0aoaEcFGuLS8V6y2M/tc2tJyIiojqw+Rt2165djqjDI2QXlAIwH7WRyWQSV0NEROSe6hRuNmzYgDvuuANKpRIbNmy47rp33323XQpzR2cKzBNmassrJK6EiIjIfdUp3IwcORK5ubkICwvDyJEja11PJpPBaDTaqza3U3UDv44cTExEROQwdQo3JpOpxsdkm7zKy8D7tmgkcSVERETuy+ZLwVevXg2dTldtuV6vx+rVq+1SlLv66uA5AECIL6+UIiIichSbw83EiRNRWFhYbXlRUREmTpxol6LcVdMQ8+XyCsUN3TuRiIiIrsPmb1khRI1X+pw9exaBgRxLcj1VV0v1uClY4kqIiIjcV50vBe/atStkMhlkMhluv/12eHld3dRoNOL06dMYNmyYQ4p0B0JcnUvK35v3uCEiInKUOn/LVl0llZ6ejqFDh8LPz8/ymkqlQmxsLO6//367F+gudBVXB2IH+iglrISIiMi91TncJCUlAQBiY2MxZswYeHt7O6wod3RtuFF7KSSshIiIyL3ZfH4kPj7eEXW4PW2ZAQAgkwFKBe9OTERE5Ch1CjchISE4ceIEQkNDERwcfN2pAwoKCuxWnDs5f8V8Az8hwKkXiIiIHKhO4ebNN9+Ev7+/5TG/nG1XWHnkRiFn74iIiBypTuHm2lNRDz/8sKNqcWsZuUUAgO5NeRk4ERGRI9l8n5sDBw7g8OHDlufffvstRo4cidmzZ0Ov19u1OHei9DK3uuoIDhERETmGzeHm3//+N06cOAEAyMzMxJgxY6DRaPD555/jueees3uB7mL/mcsAgH81D5G4EiIiIvdmc7g5ceIEunTpAgD4/PPPMWDAAHzyySdISUnBl19+ae/63MZflXcnvvaScCIiIrK/ek2/UDUz+LZt2zB8+HAAQExMDPLz8+1bnRvxVZuHN7UK95e4EiIiIvdmc7jp0aMHXn75ZaxZswY//fQT7rzzTgDA6dOnER4ebvcC3cW5y+ZLwTs14fxbREREjmRzuFmyZAkOHDiAhIQEPP/882jZsiUA4IsvvkCfPn3sXqC7KNVXAABCfFUSV0JEROTebL5DcadOnayulqqyaNEiKBScVqA2RTpzuFF72ZwniYiIyAb1np56//79OHbsGACgffv26Natm92KcjcGowlVk4KrGG6IiIgcyuZwc+HCBYwZMwY//fQTgoKCAABXrlzBwIEDsW7dOjRu3NjeNbq8yyVX7//T2E8tYSVERETuz+bDCE899RSKi4vxxx9/oKCgAAUFBThy5Ai0Wi2efvppR9To8kr0RgCAj1LBqSuIiIgczOYjN5s3b8a2bdvQrl07y7L27dtj6dKlGDJkiF2LcxdXSs1HbsoMRokrISIicn82H7kxmUxQKpXVliuVSsv9b8jaucoZwaODfCSuhIiIyP3ZHG4GDRqEqVOn4vz585Zl586dw7Rp03D77bfbtTh3cfJCMYCrl4MTERGR49gcbt555x1otVrExsaiRYsWaNGiBZo1awatVou3337bETW6PH3llAtNG/lKXAkREZH7s3nMTUxMDA4cOIDt27dbLgVv164d4uLi7F6cu9CWm2cC7xoTJG0hREREHsCmcLN+/Xps2LABer0et99+O5566ilH1eVWsgvMY2581bzJIRERkaPVOdy89957mDJlClq1agUfHx989dVXOHXqFBYtWuTI+txC1V2JlQrewI+IiMjR6vxt+8477yApKQkZGRlIT0/HRx99hHfffdeRtbkNbZn5tNRNjTQSV0JEROT+6hxuMjMzER8fb3k+duxYVFRUICcnxyGFuZMLRToAgK+q3rNdEBERUR3VOdzodDr4+l692kcul0OlUqGsrMwhhbmTqiM30cG8zw0REZGj2XQoYe7cudBorp5a0ev1WLBgAQIDAy3LFi9ebL/q3IAQApcq55ZSe3FAMRERkaPVOdzceuutyMjIsFrWp08fZGZmWp5z3qTqdBVX79oc6qeSsBIiIiLPUOdwk5aW5sAy3Ne14cZHxSM3REREjtYgrk1eunQpYmNj4e3tjV69emHv3r112m7dunWQyWQYOXKkYwu8AVXjbQBAxUvBiYiIHE7yb9v169cjMTERSUlJOHDgADp37oyhQ4fiwoUL190uKysL06dPR//+/Z1Uaf2UXDOfFE/bEREROZ7k4Wbx4sWYNGkSJk6ciPbt22PZsmXQaDRYtWpVrdsYjUaMGzcO8+fPR/PmzZ1Yre1yrpQDAKICvSWuhIiIyDNIGm70ej32799vNS+VXC5HXFwc9uzZU+t2L774IsLCwvDoo486o8wbUmESAIBcbbnElRAREXkGSe8ql5+fD6PRiPDwcKvl4eHhOH78eI3b7Ny5Ex988AHS09Pr9B46nQ46nc7yXKvVAgAMBgMMBkNtm9VL1f6u3W9WfhEAoGdssN3fz1PV1GeyP/bZOdhn52GvncNRfbZlf/UKNzt27MD777+PU6dO4YsvvkB0dDTWrFmDZs2aoV+/fvXZZZ0UFRVh/PjxWLFiBUJDQ+u0TXJyMubPn19t+datW63u2WNPqamplsdHzssAKHAmrwCbNm1yyPt5qmv7TI7DPjsH++w87LVz2LvPpaWldV7X5nDz5ZdfYvz48Rg3bhwOHjxoOSpSWFiIhQsX2vQFHhoaCoVCgby8PKvleXl5iIiIqLb+qVOnkJWVhREjRliWmUzmS629vLyQkZGBFi1aWG0za9YsJCYmWp5rtVrExMRgyJAhCAgIqHOtdWEwGJCamorBgwdDqVQCANK+OgKcOY+uzSMwfHhnu76fp6qpz2R/7LNzsM/Ow147h6P6XHXmpS5sDjcvv/wyli1bhgkTJmDdunWW5X379sXLL79s075UKhW6d++O7du3Wy7nNplM2L59OxISEqqt37ZtWxw+fNhq2Zw5c1BUVIS33noLMTEx1bZRq9VQq9XVliuVSof9cl+7b39v80+DCfyPyc4c+XdIV7HPzsE+Ow977Rz27rMt+7I53GRkZODWW2+ttjwwMBBXrlyxdXdITExEfHw8evTogZ49e2LJkiUoKSnBxIkTAQATJkxAdHQ0kpOT4e3tjQ4dOlhtHxQUBADVljcUxeXmS8E7RQf+w5pERERkDzaHm4iICJw8eRKxsbFWy3fu3Fmvy7LHjBmDixcvYt68ecjNzUWXLl2wefNmyyDj7OxsyOWSX7Feb79kXgIAaNScEZyIiMgZbP7GnTRpEqZOnYpVq1ZBJpPh/Pnz2LNnD6ZPn465c+fWq4iEhIQaT0MB/zztQ0pKSr3e01mqbtynUvAGfkRERM5gc7iZOXMmTCYTbr/9dpSWluLWW2+FWq3G9OnT8dRTTzmiRpeWV3l/m6aNfCWuhIiIyDPYHG5kMhmef/55PPvsszh58iSKi4vRvn17+Pn5OaI+lxfsq8LFIh1CNJwRnIiIyBnqPRBEpVKhffv29qzFLRmM5kvVfVSuO26IiIjIldgcbgYOHHjdCSB/+OGHGyrI3RgqzOFGyRnBiYiInMLmcNOlSxer5waDAenp6Thy5Aji4+PtVZfbKNEbATDcEBEROYvN4ebNN9+scfkLL7yA4uLiGy7InZQbjJbHvipeCk5EROQMdjuc8H//939YtWqVvXbnFvKumQk8wIfhhoiIyBnsFm727NkDb29ve+3OLRRV3p0YwHXHKREREZH92Hw44b777rN6LoRATk4Ofvvtt3rfxM9dlVaOt2kWynvcEBEROYvN4SYw0HqOJLlcjjZt2uDFF1/EkCFD7FaYO8jKLwEAqL04mJiIiMhZbAo3RqMREydORMeOHREcHOyomtzG2culAIDLpXqJKyEiIvIcNh1SUCgUGDJkSL1m//ZEVZd/Nw3RSFwJERGR57D5fEmHDh2QmZnpiFrcjq7yBn7tIwMkroSIiMhz2BxuXn75ZUyfPh3fffcdcnJyoNVqrf7QVWcKzKelVBxzQ0RE5DR1HnPz4osv4plnnsHw4cMBAHfffbfV5c1CCMhkMhiNxtp24XFMJgHg6lVTRERE5Hh1Djfz58/HE088gR9//NGR9bgVL4U5/DXyU0tcCRERkeeoc7gRwnwUYsCAAQ4rxt0cyzGfpgvzZ7ghIiJyFpsGg/Auu/VjqgyGRERE5Hg23eemdevW/xhwCgoKbqggd5JzxTy3VFSgj8SVEBEReQ6bws38+fOr3aGYaldWOSt4Iz+VxJUQERF5DpvCzYMPPoiwsDBH1eJWdBVGVFReLXVTI84tRURE5Cx1HnPD8Ta2KSwzWB4Ha5QSVkJERORZ6hxuBAfF2kRnMN+d2FspZzAkIiJyojqfljKZTI6sw+1U3bhP7aWQuBIiIiLPwnkBHORg9mUAgL6CoZCIiMiZGG4cpFhXAQDgGSkiIiLnYrhxkLOXywAA93drInElREREnoXhxkFKKo/cRAZ5S1wJERGRZ2G4cZBcrfnuxN4cUExERORUDDcOcrlUDwCQc8wNERGRUzHcOFgoZwQnIiJyKoYbB7lcYr5DcWM/hhsiIiJnYrhxkIIS82mpRgw3RERETsVw4wBCCMuM4IE+nFeKiIjImRhuHKBYZ7Q81qh4tRQREZEzMdw4gLb86ozgvuo6T99FREREdsBw4wAXtDoAQIivSuJKiIiIPA/DjQMcOa8FAITxMnAiIiKnY7hxAFH502DkjOBERETOxnDjABWVoaZjdKDElRAREXkehhsHMBjNx268FGwvERGRs/Hb1wEqTOZwo1RwYikiIiJnY7hxgKM55gHFXnK2l4iIyNn47esAVVdJ5RfrJK6EiIjI8zDcOMCvpwsAAF1igqQthIiIyAMx3DhAZKA3AFjmlyIiIiLnYbhxgMKyCgBAqzB/iSshIiLyPAw3DnDobCEAQOXF9hIRETkbv30dILaRBgAvBSciIpICw40DVN2hONBHKXElREREnofhxgEMlpv4sb1ERETOxm9fB6iwTL/A01JERETOxnDjAJdK9AB4h2IiIiIp8NvXgbzkPHJDRETkbAw3dlZ5RgoAEKThgGIiIiJnY7ixM4Pp6mNvpUK6QoiIiDxUgwg3S5cuRWxsLLy9vdGrVy/s3bu31nVXrFiB/v37Izg4GMHBwYiLi7vu+s5WeXNiAICaN/EjIiJyOsm/fdevX4/ExEQkJSXhwIED6Ny5M4YOHYoLFy7UuH5aWhoeeugh/Pjjj9izZw9iYmIwZMgQnDt3zsmV10x3zXRSMhnH3BARETmb5OFm8eLFmDRpEiZOnIj27dtj2bJl0Gg0WLVqVY3rr127FpMnT0aXLl3Qtm1brFy5EiaTCdu3b3dy5TUrMpgDTVTl5JlERETkXJKGG71ej/379yMuLs6yTC6XIy4uDnv27KnTPkpLS2EwGBASEuKoMm1SWnlaqrzCdP0ViYiIyCG8pHzz/Px8GI1GhIeHWy0PDw/H8ePH67SPGTNmICoqyiogXUun00Gn01mea7VaAIDBYIDBYKhn5TUzGAyoijRRgd523z+ZVfWV/XUs9tk52GfnYa+dw1F9tmV/koabG/XKK69g3bp1SEtLg7d3zaeBkpOTMX/+/GrLt27dCo1GY/eajCbzaSl9yRVs2rTJ7vunq1JTU6UuwSOwz87BPjsPe+0c9u5zaWlpndeVNNyEhoZCoVAgLy/PanleXh4iIiKuu+3rr7+OV155Bdu2bUOnTp1qXW/WrFlITEy0PNdqtZZByAEBATf2Af7GYDDg17XbAACRYWEYPrybXfdPZgaDAampqRg8eDCUSt5LyFHYZ+dgn52HvXYOR/W56sxLXUgablQqFbp3747t27dj5MiRAGAZHJyQkFDrdq+99hoWLFiALVu2oEePHtd9D7VaDbVaXW25Uql0yC+3vvK8lNJLwf94HMxRf4dkjX12DvbZedhr57B3n23Zl+SnpRITExEfH48ePXqgZ8+eWLJkCUpKSjBx4kQAwIQJExAdHY3k5GQAwKuvvop58+bhk08+QWxsLHJzcwEAfn5+8PPzk+xzVMktrbr8W1x3PSIiInIMycPNmDFjcPHiRcybNw+5ubno0qULNm/ebBlknJ2dDfk1E1C+99570Ov1eOCBB6z2k5SUhBdeeMGZpddIU9nRkmtveENEREROI3m4AYCEhIRaT0OlpaVZPc/KynJ8QTcgq9j8s1OTQGkLISIi8lCS38TP3QSrzD/zi/XSFkJEROShGG7srKjyMvwO0fa9EouIiIjqhuHGzvLKzAOKveScV4qIiEgKDDd2FlR5WoqTZhIREUmD4cbOjJVXgDf2r35vHSIiInI8hhs7M1WGG6WCR26IiIikwHBjZ1VHbhRytpaIiEgK/Aa2s6pwo+SAYiIiIkkw3NhZbuXVUgqGGyIiIkkw3DiIj0ohdQlEREQeieHGjoymq5Nlhgd4S1gJERGR52K4saNiXYXlcbBGJWElREREnovhxo6Kys3hRqmQQeXF1hIREUmB38B2VFhmnljKYBT/sCYRERE5CsONHekqTACAxn48JUVERCQVhhs7ulCkA8DxNkRERFJiuLEjIcyno07ll0hcCRERkediuLGjispLwbvGBEpcCRERkediuLGjisqBxN5K3sCPiIhIKgw3dlR15MaLUy8QERFJhuHGjipM5qullAq2lYiISCr8FrajqjsUc9JMIiIi6TDc2FFxuREAEOijlLgSIiIiz8VwY0cGo/m0lC9nBCciIpIMw40dVYUbjrkhIiKSDr+F7ahId3XiTCIiIpIGw40dnc4vlboEIiIij8dwY0cRAWoAVyfQJCIiIudjuLEjQ+UdimNCfCSuhIiIyHMx3NiRngOKiYiIJMdvYTs6f6UMAMMNERGRlPgtbEfaMvPVUkYTx9wQERFJheHGjkL9VQAAX5WXxJUQERF5LoYbOzJWDij282a4ISIikgrDjR1VmMzhxosTZxIREUmG4caOjJXhhrOCExERSYfhxo4qGG6IiIgkx3BjR39dNl8KztNSRERE0mG4cQAhpK6AiIjIczHc2JGf2nyVVIAPr5YiIiKSCsONHYnKQzZyGU9LERERSYXhxo6MDDdERESSY7ixo8qLpcDxxERERNJhuLEjy2kpphsiIiLJMNzYUdVN/HhaioiISDoMN3bE01JERETSY7ixE3HNzW1kPHJDREQkGYYbO6k6JQUACoYbIiIiyTDc2Mk12YanpYiIiCTEcGMnJp6WIiIiahAYbuxEZzBZHqsUDDdERERSYbixE73xarhRKxUSVkJEROTZGG7spOpqKRk4JTgREZGUGG7s5Oq8UhIXQkRE5OEYbuzEcndiiesgIiLydPwutpOqi6V4oRQREZG0GG7spOrIDcMNERGRtBpEuFm6dCliY2Ph7e2NXr16Ye/evddd//PPP0fbtm3h7e2Njh07YtOmTU6qtHaWMTcS10FEROTpJP8uXr9+PRITE5GUlIQDBw6gc+fOGDp0KC5cuFDj+rt378ZDDz2ERx99FAcPHsTIkSMxcuRIHDlyxMmVW7NcLcUjN0RERJKSPNwsXrwYkyZNwsSJE9G+fXssW7YMGo0Gq1atqnH9t956C8OGDcOzzz6Ldu3a4aWXXkK3bt3wzjvvOLlya2V6831uJG8oERGRh/OS8s31ej3279+PWbNmWZbJ5XLExcVhz549NW6zZ88eJCYmWi0bOnQovvnmmxrX1+l00Ol0ludarRYAYDAYYDAYbvATXHXs/BUA5iM39twvVVfVX/bZsdhn52CfnYe9dg5H9dmW/UkabvLz82E0GhEeHm61PDw8HMePH69xm9zc3BrXz83NrXH95ORkzJ8/v9ryrVu3QqPR1LPy6nKKAKVcgW6hAqmpqXbbL9WOfXYO9tk52GfnYa+dw959Li0trfO6koYbZ5g1a5bVkR6tVouYmBgMGTIEAQEBdn2vSQYDUlNTMXjwYCiVSrvum64ysM9OwT47B/vsPOy1cziqz1VnXupC0nATGhoKhUKBvLw8q+V5eXmIiIiocZuIiAib1ler1VCr1dWWK5VKh/1yO3LfdBX77Bzss3Owz87DXjuHvftsy74kHf+qUqnQvXt3bN++3bLMZDJh+/bt6N27d43b9O7d22p9wHzoq7b1iYiIyLNIfloqMTER8fHx6NGjB3r27IklS5agpKQEEydOBABMmDAB0dHRSE5OBgBMnToVAwYMwBtvvIE777wT69atw2+//Ybly5dL+TGIiIiogZA83IwZMwYXL17EvHnzkJubiy5dumDz5s2WQcPZ2dmQy68eYOrTpw8++eQTzJkzB7Nnz0arVq3wzTffoEOHDlJ9BCIiImpAJA83AJCQkICEhIQaX0tLS6u2bNSoURg1apSDqyIiIiJXxHvOERERkVthuCEiIiK3wnBDREREboXhhoiIiNwKww0RERG5FYYbIiIicisMN0RERORWGG6IiIjIrTDcEBERkVtpEHcodiYhBADbpk6vK4PBgNLSUmi1Ws4460Dss3Owz87BPjsPe+0cjupz1fd21ff49XhcuCkqKgIAxMTESFwJERER2aqoqAiBgYHXXUcm6hKB3IjJZML58+fh7+8PmUxm131rtVrExMTgr7/+QkBAgF33TVexz87BPjsH++w87LVzOKrPQggUFRUhKirKakLtmnjckRu5XI4mTZo49D0CAgL4H44TsM/OwT47B/vsPOy1cziiz/90xKYKBxQTERGRW2G4ISIiIrfCcGNHarUaSUlJUKvVUpfi1thn52CfnYN9dh722jkaQp89bkAxERERuTceuSEiIiK3wnBDREREboXhhoiIiNwKww0RERG5FYYbGy1duhSxsbHw9vZGr169sHfv3uuu//nnn6Nt27bw9vZGx44dsWnTJidV6tps6fOKFSvQv39/BAcHIzg4GHFxcf/490Jmtv4+V1m3bh1kMhlGjhzp2ALdhK19vnLlCqZMmYLIyEio1Wq0bt2a/3bUga19XrJkCdq0aQMfHx/ExMRg2rRpKC8vd1K1runnn3/GiBEjEBUVBZlMhm+++eYft0lLS0O3bt2gVqvRsmVLpKSkOLxOCKqzdevWCZVKJVatWiX++OMPMWnSJBEUFCTy8vJqXH/Xrl1CoVCI1157TRw9elTMmTNHKJVKcfjwYSdX7lps7fPYsWPF0qVLxcGDB8WxY8fEww8/LAIDA8XZs2edXLlrsbXPVU6fPi2io6NF//79xT333OOcYl2YrX3W6XSiR48eYvjw4WLnzp3i9OnTIi0tTaSnpzu5ctdia5/Xrl0r1Gq1WLt2rTh9+rTYsmWLiIyMFNOmTXNy5a5l06ZN4vnnnxdfffWVACC+/vrr666fmZkpNBqNSExMFEePHhVvv/22UCgUYvPmzQ6tk+HGBj179hRTpkyxPDcajSIqKkokJyfXuP7o0aPFnXfeabWsV69e4t///rdD63R1tvb57yoqKoS/v7/46KOPHFWiW6hPnysqKkSfPn3EypUrRXx8PMNNHdja5/fee080b95c6PV6Z5XoFmzt85QpU8SgQYOsliUmJoq+ffs6tE53Updw89xzz4mbb77ZatmYMWPE0KFDHViZEDwtVUd6vR779+9HXFycZZlcLkdcXBz27NlT4zZ79uyxWh8Ahg4dWuv6VL8+/11paSkMBgNCQkIcVabLq2+fX3zxRYSFheHRRx91Rpkurz593rBhA3r37o0pU6YgPDwcHTp0wMKFC2E0Gp1VtsupT5/79OmD/fv3W05dZWZmYtOmTRg+fLhTavYUUn0PetzEmfWVn58Po9GI8PBwq+Xh4eE4fvx4jdvk5ubWuH5ubq7D6nR19enz382YMQNRUVHV/oOiq+rT5507d+KDDz5Aenq6Eyp0D/Xpc2ZmJn744QeMGzcOmzZtwsmTJzF58mQYDAYkJSU5o2yXU58+jx07Fvn5+ejXrx+EEKioqMATTzyB2bNnO6Nkj1Hb96BWq0VZWRl8fHwc8r48ckNu5ZVXXsG6devw9ddfw9vbW+py3EZRURHGjx+PFStWIDQ0VOpy3JrJZEJYWBiWL1+O7t27Y8yYMXj++eexbNkyqUtzK2lpaVi4cCHeffddHDhwAF999RU2btyIl156SerSyA545KaOQkNDoVAokJeXZ7U8Ly8PERERNW4TERFh0/pUvz5Xef311/HKK69g27Zt6NSpkyPLdHm29vnUqVPIysrCiBEjLMtMJhMAwMvLCxkZGWjRooVji3ZB9fl9joyMhFKphEKhsCxr164dcnNzodfroVKpHFqzK6pPn+fOnYvx48fjscceAwB07NgRJSUlePzxx/H8889DLuf/+9tDbd+DAQEBDjtqA/DITZ2pVCp0794d27dvtywzmUzYvn07evfuXeM2vXv3tlofAFJTU2tdn+rXZwB47bXX8NJLL2Hz5s3o0aOHM0p1abb2uW3btjh8+DDS09Mtf+6++24MHDgQ6enpiImJcWb5LqM+v899+/bFyZMnLeERAE6cOIHIyEgGm1rUp8+lpaXVAkxVoBScctFuJPsedOhwZTezbt06oVarRUpKijh69Kh4/PHHRVBQkMjNzRVCCDF+/Hgxc+ZMy/q7du0SXl5e4vXXXxfHjh0TSUlJvBS8Dmzt8yuvvCJUKpX44osvRE5OjuVPUVGRVB/BJdja57/j1VJ1Y2ufs7Ozhb+/v0hISBAZGRniu+++E2FhYeLll1+W6iO4BFv7nJSUJPz9/cWnn34qMjMzxdatW0WLFi3E6NGjpfoILqGoqEgcPHhQHDx4UAAQixcvFgcPHhRnzpwRQggxc+ZMMX78eMv6VZeCP/vss+LYsWNi6dKlvBS8IXr77bdF06ZNhUqlEj179hS//PKL5bUBAwaI+Ph4q/U/++wz0bp1a6FSqcTNN98sNm7c6OSKXZMtfb7pppsEgGp/kpKSnF+4i7H19/laDDd1Z2ufd+/eLXr16iXUarVo3ry5WLBggaioqHBy1a7Hlj4bDAbxwgsviBYtWghvb28RExMjJk+eLC5fvuz8wl3Ijz/+WOO/t1W9jY+PFwMGDKi2TZcuXYRKpRLNmzcXH374ocPrlAnB429ERETkPjjmhoiIiNwKww0RERG5FYYbIiIicisMN0RERORWGG6IiIjIrTDcEBERkVthuCEiIiK3wnBDRFZSUlIQFBQkdRn1JpPJ8M0331x3nYcffhgjR450Sj1E5HwMN0Ru6OGHH4ZMJqv25+TJk1KXhpSUFEs9crkcTZo0wcSJE3HhwgW77D8nJwd33HEHACArKwsymQzp6elW67z11ltISUmxy/vV5oUXXrB8ToVCgZiYGDz++OMoKCiwaT8MYkS246zgRG5q2LBh+PDDD62WNW7cWKJqrAUEBCAjIwMmkwmHDh3CxIkTcf78eWzZsuWG9/1Ps8cDQGBg4A2/T13cfPPN2LZtG4xGI44dO4ZHHnkEhYWFWL9+vVPen8hT8cgNkZtSq9WIiIiw+qNQKLB48WJ07NgRvr6+iImJweTJk1FcXFzrfg4dOoSBAwfC398fAQEB6N69O3777TfL6zt37kT//v3h4+ODmJgYPP300ygpKblubTKZDBEREYiKisIdd9yBp59+Gtu2bUNZWRlMJhNefPFFNGnSBGq1Gl26dMHmzZst2+r1eiQkJCAyMhLe3t646aabkJycbLXvqtNSzZo1AwB07doVMpkMt912GwDroyHLly9HVFSU1SzcAHDPPffgkUcesTz/9ttv0a1bN3h7e6N58+aYP38+Kioqrvs5vby8EBERgejoaMTFxWHUqFFITU21vG40GvHoo4+iWbNm8PHxQZs2bfDWW29ZXn/hhRfw0Ucf4dtvv7UcBUpLSwMA/PXXXxg9ejSCgoIQEhKCe+65B1lZWdeth8hTMNwQeRi5XI7//ve/+OOPP/DRRx/hhx9+wHPPPVfr+uPGjUOTJk2wb98+7N+/HzNnzoRSqQQAnDp1CsOGDcP999+P33//HevXr8fOnTuRkJBgU00+Pj4wmUyoqKjAW2+9hTfeeAOvv/46fv/9dwwdOhR33303/vzzTwDAf//7X2zYsAGfffYZMjIysHbtWsTGxta437179wIAtm3bhpycHHz11VfV1hk1ahQuXbqEH3/80bKsoKAAmzdvxrhx4wAAO3bswIQJEzB16lQcPXoU77//PlJSUrBgwYI6f8asrCxs2bIFKpXKssxkMqFJkyb4/PPPcfToUcybNw+zZ8/GZ599BgCYPn06Ro8ejWHDhiEnJwc5OTno06cPDAYDhg4dCn9/f+zYsQO7du2Cn58fhg0bBr1eX+eaiNyWw6fmJCKni4+PFwqFQvj6+lr+PPDAAzWu+/nnn4tGjRpZnn/44YciMDDQ8tzf31+kpKTUuO2jjz4qHn/8catlO3bsEHK5XJSVldW4zd/3f+LECdG6dWvRo0cPIYQQUVFRYsGCBVbb3HLLLWLy5MlCCCGeeuopMWjQIGEymWrcPwDx9ddfCyGEOH36tAAgDh48aLXO32c0v+eee8Qjjzxief7++++LqKgoYTQahRBC3H777WLhwoVW+1izZo2IjIyssQYhhEhKShJyuVz4+voKb29vy+zJixcvrnUbIYSYMmWKuP/++2utteq927RpY9UDnU4nfHx8xJYtW667fyJPwDE3RG5q4MCBeO+99yzPfX19AZiPYiQnJ+P48ePQarWoqKhAeXk5SktLodFoqu0nMTERjz32GNasWWM5tdKiRQsA5lNWv//+O9auXWtZXwgBk8mE06dPo127djXWVlhYCD8/P5hMJpSXl6Nfv35YuXIltFotzp8/j759+1qt37dvXxw6dAiA+ZTS4MGD0aZNGwwbNgx33XUXhgwZckO9GjduHCZNmoR3330XarUaa9euxYMPPgi5XG75nLt27bI6UmM0Gq/bNwBo06YNNmzYgPLycnz88cdIT0/HU089ZbXO0qVLsWrVKmRnZ6OsrAx6vR5dunS5br2HDh3CyZMn4e/vb7W8vLwcp06dqkcHiNwLww2Rm/L19UXLli2tlmVlZeGuu+7Ck08+iQULFiAkJAQ7d+7Eo48+Cr1eX+OX9AsvvICxY8di48aN+P7775GUlIR169bh3nvvRXFxMf7973/j6aefrrZd06ZNa63N398fBw4cgFwuR2RkJHx8fAAAWq32Hz9Xt27dcPr0aXz//ffYtm0bRo8ejbi4OHzxxRf/uG1tRowYASEENm7ciFtuuQU7duzAm2++aXm9uLgY8+fPx3333VdtW29v71r3q1KpLH8Hr7zyCu68807Mnz8fL730EgBg3bp1mD59Ot544w307t0b/v7+WLRoEX799dfr1ltcXIzu3btbhcoqDWXQOJGUGG6IPMj+/fthMpnwxhtvWI5KVI3vuJ7WrVujdevWmDZtGh566CF8+OGHuPfee9GtWzccPXq0Woj6J3K5vMZtAgICEBUVhV27dmHAgAGW5bt27ULPnj2t1hszZgzGjBmDBx54AMOGDUNBQQFCQkKs9lc1vsVoNF63Hm9vb9x3331Yu3YtTp48iTZt2qBbt26W17t164aMjAybP+ffzZkzB4MGDcKTTz5p+Zx9+vTB5MmTLev8/ciLSqWqVn+3bt2wfv16hIWFISAg4IZqInJHHFBM5EFatmwJg8GAt99+G5mZmVizZg2WLVtW6/plZWVISEhAWloazpw5g127dmHfvn2W000zZszA7t27kZCQgPT0dPz555/49ttvbR5QfK1nn30Wr776KtavX4+MjAzMnDkT6enpmDp1KgBg8eLF+PTTT3H8+HGcOHECn3/+OSIiImq88WBYWBh8fHywefNm5OXlobCwsNb3HTduHDZu3IhVq1ZZBhJXmTdvHlavXo358+fjjz/+wLFjx7Bu3TrMmTPHps/Wu3dvdOrUCQsXLgQAtGrVCr/99hu2bNmCEydOYO7cudi3b5/VNrGxsfj999+RkZGB/Px8GAwGjBs3DqGhobjnnnuwY8cOnD59GmlpaXj66adx9uxZm2oicktSD/ohIvuraRBqlcWLF4vIyEjh4+Mjhg4dKlavXi0AiMuXLwshrAf86nQ68eCDD4qYmBihUqlEVFSUSEhIsBosvHfvXjF48GDh5+cnfH19RadOnaoNCL7W3wcU/53RaBQvvPCCiI6OFkqlUnTu3Fl8//33lteXL18uunTpInx9fUVAQIC4/fbbxYEDByyv45oBxUIIsWLFChETEyPkcrkYMGBArf0xGo0iMjJSABCnTp2qVtfmzZtFnz59hI+PjwgICBA9e/YUy5cvr/VzJCUlic6dO1db/umnnwq1Wi2ys7NFeXm5ePjhh0VgYKAICgoSTz75pJg5c6bVdhcuXLD0F4D48ccfhRBC5OTkiAkTJojQ0FChVqtF8+bNxaRJk0RhYWGtNRF5CpkQQkgbr4iIiIjsh6eliIiIyK0w3BAREZFbYbghIiIit8JwQ0RERG6F4YaIiIjcCsMNERERuRWGGyIiInIrDDdERETkVhhuiIiIyK0w3BAREZFbYbghIiIit8JwQ0RERG7l/wHfgSkrrMzc6wAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "AUC ROC: 0.9420113727999999\n"
          ]
        }
      ],
      "source": [
        "#построение ROC-кривой и AUC ROC\n",
        "from sklearn.metrics import roc_curve, auc\n",
        "\n",
        "fpr, tpr, thresholds = roc_curve(y_test, y_score)\n",
        "plt.figure(figsize=(8, 6))\n",
        "plt.plot(fpr, tpr)\n",
        "plt.grid()\n",
        "plt.xlabel('False Positive Rate')\n",
        "plt.ylabel('True Positive Rate')\n",
        "plt.title('ROC')\n",
        "plt.savefig('roc_curve.png', dpi=300, bbox_inches='tight')\n",
        "plt.show()\n",
        "print('AUC ROC:', auc(fpr, tpr))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MsM3ew3d1FYq"
      },
      "source": [
        "### 11) Сделали выводы по результатам применения рекуррентной нейронной сети для решения задачи определения тональности текста.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xxFO4CXbIG88"
      },
      "source": [
        "Таблица1:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xvoivjuNFlEf"
      },
      "source": [
        "| Модель   | Количество настраиваемых параметров | Количество эпох обучения | Качество классификации тестовой выборки |\n",
        "|----------|-------------------------------------|---------------------------|-----------------------------------------|\n",
        "| Рекуррентная | 184 897                              | 5                        | accuracy:0.8659     ; loss:0.3349     ; AUC ROC:0.9420                                   |\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YctF8h_sIB-P"
      },
      "source": [
        "#### По результатам применения рекуррентной нейронной сети, а также по данным таблицы 1 можно сделать вывод, что модель хорошо справилась с задачей определения тональности текста. Показатель accuracy = 0.8659 превышает требуемый порог 0.8. Значение AUC ROC = 0.9420 (> 0.9) говорит о высокой способности модели различать два класса (положительные и отрицательные отзывы). Модель показала хорошие результаты по метрикам precision и recall: для негативных отзывов precision = 0.91, recall = 0.82; для позитивных отзывов precision = 0.83, recall = 0.92."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}