# Отчёт по лабораторной работе №4 **Ли Тэ Хо, Синявский Степан — А-02-22** **Бригада 3** --- ## Задание 1 ### 1) В среде Google Colab создали новый блокнот (notebook). Импортировали необходимые для работы библиотеки и модули. Настроили блокнот для работы с аппаратным ускорителем GPU. ```python # импорт модулей import os os.chdir('/content/drive/MyDrive/Colab Notebooks/is_lab4') from tensorflow import keras from tensorflow.keras import layers from tensorflow.keras.models import Sequential import matplotlib.pyplot as plt import numpy as np ``` ```python import tensorflow as tf device_name = tf.test.gpu_device_name() if device_name != '/device:GPU:0': raise SystemError('GPU device not found') print('Found GPU at: {}'.format(device_name)) ``` ``` Found GPU at: /device:GPU:0 ``` ### 2) Загрузили набор данных IMDb, содержащий оцифрованные отзывы на фильмы, размеченные на два класса: позитивные и негативные. При загрузке набора данных параметр seed выбрали равным значению (4k – 1)=11, где k=3 – номер бригады. Вывели размеры полученных обучающих и тестовых массивов данных. ```python # загрузка датасета from keras.datasets import imdb vocabulary_size = 5000 index_from = 3 (X_train, y_train), (X_test, y_test) = imdb.load_data( path="imdb.npz", num_words=vocabulary_size, skip_top=0, maxlen=None, seed=11, start_char=1, oov_char=2, index_from=index_from ) # вывод размерностей print('Shape of X train:', X_train.shape) print('Shape of y train:', y_train.shape) print('Shape of X test:', X_test.shape) print('Shape of y test:', y_test.shape) ``` ``` Shape of X train: (25000,) Shape of y train: (25000,) Shape of X test: (25000,) Shape of y test: (25000,) ``` ### 3) Вывели один отзыв из обучающего множества в виде списка индексов слов. Преобразовали список индексов в текст и вывели отзыв в виде текста. Вывели длину отзыва. Вывели метку класса данного отзыва и название класса (1 – Positive, 0 – Negative). ```python # создание словаря для перевода индексов в слова # заргузка словаря "слово:индекс" word_to_id = imdb.get_word_index() # уточнение словаря word_to_id = {key:(value + index_from) for key,value in word_to_id.items()} word_to_id[""] = 0 word_to_id[""] = 1 word_to_id[""] = 2 word_to_id[""] = 3 # создание обратного словаря "индекс:слово" id_to_word = {value:key for key,value in word_to_id.items()} ``` ```python print(X_train[26]) print('len:',len(X_train[26])) ``` ``` [1, 2489, 723, 2, 9, 399, 2301, 11, 551, 2, 29, 47, 1391, 6, 1692, 15, 29, 70, 361, 8, 97, 35, 3258, 40, 6, 2, 106, 42, 2, 4298, 64, 8, 28, 15, 3258, 796, 2, 11, 6, 275, 1622, 21, 50, 26, 148, 33, 27, 2301, 2, 15, 81, 24, 40, 42, 2, 7, 27, 4646, 5, 80, 81, 845, 12, 304, 8, 67, 15, 29, 152, 3115, 103, 6, 1196, 2, 15, 238, 28, 1894, 27, 2, 2489, 2, 1068, 8, 2181, 27, 1692, 23, 309, 17, 873, 183, 140, 2357, 355, 5, 29, 9, 2, 83, 6, 2699, 1765, 2, 625, 2691, 1229, 80, 516, 10, 10, 11, 2, 279, 12, 286, 141, 6, 52, 326, 8, 796, 106, 4, 2, 132, 11, 4, 172, 1269, 13, 296, 4, 2223, 994, 7, 4, 2223, 5, 3176, 7, 4, 2223, 50, 186, 8, 30, 64, 38, 111, 102, 44, 551, 2, 5, 4, 4616, 3388, 302, 12, 70, 28, 23, 4, 406, 648, 15, 31, 415, 144, 30, 93, 8, 4325, 11, 6, 289, 42, 689, 251, 810, 146, 24, 252, 51, 148, 1893, 18, 4, 20, 1029, 17, 68, 2436, 819, 18, 4, 2, 132, 21, 76, 7, 12, 9, 38, 729, 8, 4, 2223, 102, 15, 12, 566, 30, 2691, 2, 190, 4, 2, 132, 218, 60, 754, 17, 52, 17, 4, 249, 7, 4, 2223, 2355, 10, 10, 1371, 112, 1905, 4981, 4, 2, 132, 47, 450, 85, 712, 15, 66, 1487, 4, 3129, 7, 4, 20, 6, 194, 1834, 13, 28, 9, 19, 2, 2, 11, 4, 485, 240, 141, 6, 2, 1995, 15, 24, 64, 81, 13, 24, 459, 44, 27, 2073, 13, 165, 3663, 18, 12, 696, 177, 1066, 1083, 2, 5, 2, 1602, 26, 220, 17, 78, 507, 38, 1904, 5, 753, 36, 983, 551, 11, 192, 225, 55, 117, 8, 79, 2229, 44, 137, 149, 4, 2, 132, 4, 816, 475, 24, 55, 906, 4, 168, 475, 13, 62, 1634, 76, 7, 12, 17, 2, 4, 114, 475, 727, 4, 206, 475, 50, 218, 101, 444, 14, 9, 31, 8, 798, 10, 10, 2994, 13, 296, 4, 2, 132, 2864, 6, 1039, 7, 4, 736, 1067, 750, 2, 390, 163, 538, 137, 24, 35, 1557, 55, 400, 4, 2, 4, 20, 475, 4, 128, 4, 3179, 2, 4, 493, 569, 220, 32, 7, 68, 3734, 19, 4, 2, 132, 637, 202, 12, 6, 55, 2, 470, 457, 23, 61, 3179, 675, 2407] len: 413 ``` ```python review_as_text = ' '.join(id_to_word[id] for id in X_train[26]) print(review_as_text) print('len:',len(review_as_text)) ``` ``` professor paul is doing research in matter he has developed a machine that he can use to make an object like a watch or disappear only to have that object re in a different location but there are those at his research that do not like or of his experiments and will do whatever it takes to see that he doesn't succeed after a failed that might have saved his professor decides to test his machine on himself as expected things go horribly wrong and he is into a heavily scared whose mere touch will kill br br in maybe it wasn't such a good idea to re watch the man in the same week i watched the fly return of the fly and curse of the fly there seems to be only so many movies about matter and the potentially horrendous effects it can have on the human body that one person should be made to endure in a three or four day period i'm not sure what those responsible for the movie list as their source material for the man but much of it is so similar to the fly movies that it cannot be mere however the man isn't even nearly as good as the worst of the fly trilogy br br besides being terribly unoriginal the man has several other problems that really hurt the enjoyment of the movie a big issue i have is with in the lead he's such a ass that not only do i not care about his suffering i actually root for it supporting cast members mary and allen are almost as bad they're so bland and dull they hardly matter in fact there's very little to get excited about while watching the man the soundtrack – not very memorable the look – i would describe much of it as the plot – predictable the action – there isn't any overall this is one to avoid br br fortunately i watched the man via a copy of the mystery science theater episode funny stuff while not an absolute very often the the movie – the better the mst3k the guys hit almost all of their marks with the man i'll give it a very 4 5 on my mst3k rating scale len: 2113 ``` ### 4) Вывели максимальную и минимальную длину отзыва в обучающем множестве. ```python print('MAX Len: ',len(max(X_train, key=len))) print('MIN Len: ',len(min(X_train, key=len))) ``` ``` MAX Len: 2494 MIN Len: 11 ``` ### 5) Провели предобработку данных. Выбрали единую длину, к которой будут приведены все отзывы. Короткие отзывы дополнили спецсимволами, а длинные обрезали до выбранной длины. ```python # предобработка данных from tensorflow.keras.utils import pad_sequences max_words = 500 X_train = pad_sequences(X_train, maxlen=max_words, value=0, padding='pre', truncating='post') X_test = pad_sequences(X_test, maxlen=max_words, value=0, padding='pre', truncating='post') ``` ### 6) Повторили пункт 4. ```python print('MAX Len: ',len(max(X_train, key=len))) print('MIN Len: ',len(min(X_train, key=len))) ``` ``` MAX Len: 500 MIN Len: 500 ``` ### 7) Повторили пункт 3. Сделали вывод о том, как отзыв преобразовался после предобработки. ```python print(X_train[26]) print('len:',len(X_train[26])) ``` ``` [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2489 723 2 9 399 2301 11 551 2 29 47 1391 6 1692 15 29 70 361 8 97 35 3258 40 6 2 106 42 2 4298 64 8 28 15 3258 796 2 11 6 275 1622 21 50 26 148 33 27 2301 2 15 81 24 40 42 2 7 27 4646 5 80 81 845 12 304 8 67 15 29 152 3115 103 6 1196 2 15 238 28 1894 27 2 2489 2 1068 8 2181 27 1692 23 309 17 873 183 140 2357 355 5 29 9 2 83 6 2699 1765 2 625 2691 1229 80 516 10 10 11 2 279 12 286 141 6 52 326 8 796 106 4 2 132 11 4 172 1269 13 296 4 2223 994 7 4 2223 5 3176 7 4 2223 50 186 8 30 64 38 111 102 44 551 2 5 4 4616 3388 302 12 70 28 23 4 406 648 15 31 415 144 30 93 8 4325 11 6 289 42 689 251 810 146 24 252 51 148 1893 18 4 20 1029 17 68 2436 819 18 4 2 132 21 76 7 12 9 38 729 8 4 2223 102 15 12 566 30 2691 2 190 4 2 132 218 60 754 17 52 17 4 249 7 4 2223 2355 10 10 1371 112 1905 4981 4 2 132 47 450 85 712 15 66 1487 4 3129 7 4 20 6 194 1834 13 28 9 19 2 2 11 4 485 240 141 6 2 1995 15 24 64 81 13 24 459 44 27 2073 13 165 3663 18 12 696 177 1066 1083 2 5 2 1602 26 220 17 78 507 38 1904 5 753 36 983 551 11 192 225 55 117 8 79 2229 44 137 149 4 2 132 4 816 475 24 55 906 4 168 475 13 62 1634 76 7 12 17 2 4 114 475 727 4 206 475 50 218 101 444 14 9 31 8 798 10 10 2994 13 296 4 2 132 2864 6 1039 7 4 736 1067 750 2 390 163 538 137 24 35 1557 55 400 4 2 4 20 475 4 128 4 3179 2 4 493 569 220 32 7 68 3734 19 4 2 132 637 202 12 6 55 2 470 457 23 61 3179 675 2407] len: 500 ``` ```python review_as_text = ' '.join(id_to_word[id] for id in X_train[26]) print(review_as_text) print('len:',len(review_as_text)) ``` ``` professor paul is doing research in matter he has developed a machine that he can use to make an object like a watch or disappear only to have that object re in a different location but there are those at his research that do not like or of his experiments and will do whatever it takes to see that he doesn't succeed after a failed that might have saved his professor decides to test his machine on himself as expected things go horribly wrong and he is into a heavily scared whose mere touch will kill br br in maybe it wasn't such a good idea to re watch the man in the same week i watched the fly return of the fly and curse of the fly there seems to be only so many movies about matter and the potentially horrendous effects it can have on the human body that one person should be made to endure in a three or four day period i'm not sure what those responsible for the movie list as their source material for the man but much of it is so similar to the fly movies that it cannot be mere however the man isn't even nearly as good as the worst of the fly trilogy br br besides being terribly unoriginal the man has several other problems that really hurt the enjoyment of the movie a big issue i have is with in the lead he's such a ass that not only do i not care about his suffering i actually root for it supporting cast members mary and allen are almost as bad they're so bland and dull they hardly matter in fact there's very little to get excited about while watching the man the soundtrack – not very memorable the look – i would describe much of it as the plot – predictable the action – there isn't any overall this is one to avoid br br fortunately i watched the man via a copy of the mystery science theater episode funny stuff while not an absolute very often the the movie – the better the mst3k the guys hit almost all of their marks with the man i'll give it a very 4 5 on my mst3k rating scale len: 2635 ``` #### После обработки в начало отзыва добавилось необходимое количество токенов , чтобы отзыв был длинной в 500 индексов. ### 8) Вывели предобработанные массивы обучающих и тестовых данных и их размерности. ```python # вывод данных print('X train: \n',X_train) print('X train: \n',X_test) # вывод размерностей print('Shape of X train:', X_train.shape) print('Shape of X test:', X_test.shape) ``` ``` X train: [[ 0 0 0 ... 6 2 2] [ 0 0 0 ... 10 10 2] [ 1 14 22 ... 171 153 303] ... [ 0 0 0 ... 17 2199 1262] [ 0 0 0 ... 606 5 1356] [ 0 0 0 ... 1026 5 804]] X train: [[ 0 0 0 ... 10 10 2] [ 0 0 0 ... 43 1044 710] [ 0 0 0 ... 35 744 23] ... [ 0 0 0 ... 184 1543 616] [ 0 0 0 ... 38 2 78] [ 0 0 0 ... 5 2 2]] Shape of X train: (25000, 500) Shape of X test: (25000, 500) ``` ### 9) Реализовали модель рекуррентной нейронной сети, состоящей из слоев Embedding, LSTM, Dropout, Dense, и обучили ее на обучающих данных с выделением части обучающих данных в качестве валидационных. Вывели информацию об архитектуре нейронной сети. Добились качества обучения по метрике accuracy не менее 0.8. ```python embed_dim = 32 lstm_units = 64 model = Sequential() model.add(layers.Embedding(input_dim=vocabulary_size, output_dim=embed_dim, input_length=max_words, input_shape=(max_words,))) model.add(layers.LSTM(lstm_units)) model.add(layers.Dropout(0.5)) model.add(layers.Dense(1, activation='sigmoid')) model.summary() ``` **Model: "sequential"** | Layer (type) | Output Shape | Param # | | ----------------------- | --------------- | ------: | | embedding_4 (Embedding) | (None, 500, 32) | 160,000 | | lstm_4 (LSTM) | (None, 64) | 24,832 | | dropout_4 (Dropout) | (None, 64) | 0 | | dense_4 (Dense) | (None, 1) | 65 | **Total params:** 184,897 (722.25 KB) **Trainable params:** 184,897 (722.25 KB) **Non-trainable params:** 0 (0.00 B) ```python # компилируем и обучаем модель batch_size = 64 epochs = 3 model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"]) model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.2) ``` ``` Epoch 1/3 313/313 ━━━━━━━━━━━━━━━━━━━━ 13s 23ms/step - accuracy: 0.6613 - loss: 0.5831 - val_accuracy: 0.8470 - val_loss: 0.3631 Epoch 2/3 313/313 ━━━━━━━━━━━━━━━━━━━━ 18s 25ms/step - accuracy: 0.8749 - loss: 0.3133 - val_accuracy: 0.7728 - val_loss: 0.5550 Epoch 3/3 313/313 ━━━━━━━━━━━━━━━━━━━━ 7s 21ms/step - accuracy: 0.8655 - loss: 0.3285 - val_accuracy: 0.8696 - val_loss: 0.3508 ``` ```python test_loss, test_acc = model.evaluate(X_test, y_test) print(f"\nTest accuracy: {test_acc}") ``` ``` 782/782 ━━━━━━━━━━━━━━━━━━━━ 9s 11ms/step - accuracy: 0.8611 - loss: 0.3604 Test accuracy: 0.8602399826049805 ``` ### 10) Оценили качество обучения на тестовых данных: ### - вывели значение метрики качества классификации на тестовых данных ### - вывели отчет о качестве классификации тестовой выборки ### - построили ROC-кривую по результату обработки тестовой выборки и вычислили площадь под ROC-кривой (AUC ROC) ```python #значение метрики качества классификации на тестовых данных print(f"\nTest accuracy: {test_acc}") ``` ``` Test accuracy: 0.8602399826049805 ``` ```python #отчет о качестве классификации тестовой выборки y_score = model.predict(X_test) y_pred = [1 if y_score[i,0]>=0.5 else 0 for i in range(len(y_score))] from sklearn.metrics import classification_report print(classification_report(y_test, y_pred, labels = [0, 1], target_names=['Negative', 'Positive'])) ``` ``` precision recall f1-score support Negative 0.82 0.92 0.87 12500 Positive 0.91 0.80 0.85 12500 accuracy 0.86 25000 macro avg 0.86 0.86 0.86 25000 weighted avg 0.86 0.86 0.86 25000 ``` ```python #построение ROC-кривой и AUC ROC from sklearn.metrics import roc_curve, auc fpr, tpr, thresholds = roc_curve(y_test, y_score) plt.plot(fpr, tpr) plt.grid() plt.xlabel('False Positive Rate') plt.ylabel('True Positive Rate') plt.title('ROC') plt.show() print('AUC ROC:', auc(fpr, tpr)) ``` ![picture](images/1.1.png) ``` AUC ROC: 0.9378295648 ``` ### 11) Сделали выводы по результатам применения рекуррентной нейронной сети для решения задачи определения тональности текста. Таблица1: | Модель | Количество настраиваемых параметров | Количество эпох обучения | Качество классификации тестовой выборки | |----------|-------------------------------------|---------------------------|-----------------------------------------| | Рекуррентная | 184 897 | 3 | accuracy:0.860 ; loss:0.3604 ; AUC ROC:0.9378 | #### По полученной таблице можно сделать вывод о хорошей способности рекуррентной нейронной сети определять тональности текста, это подтверждает показатель accuracy = 0.860, который превышает заданный порог в 0.8 #### Значение AUC ROC = 0.9378 (> 0.9) показывает, что нейронная сеть уверенно различает тональности (правильно ставит позитивный класс ниже негативного).