深度学习补充一

发表于 | 分类于 | 热度: ℃ |
字数统计: 1.4k | 阅读时长 ≈ 6

深度学习补充

  1. 理解卷积神经网络
  2. 使用数据增强来降低过拟合
  3. 使用预训练的卷积神经网络进行特征提取
  4. 微调预训练的卷积神经网络
  5. 将卷积神经网络学习到的内容以及如何做出分类决策可视化

理解卷积神经网络

机器学习通用工作流程

  1. 学会去定义所面对的问题
  2. 选择衡量成功的指标
  3. 确定评估方法;数据量大可以使用留出验证集;验证样本量太少,无法确保可靠性,使用K折交叉验证;数量少,模型评估又需要非常准确可以使用重复的K折验证.
  4. 准备数据,(1)数据格式张量化(2)取值应该都在比较小的值范围之间(3)不同的特征具有不同的取值范围，数据就应该标准化。(4)对于小数据可能需要做特征工程
  5. 一般二分类使用 sigmoid,损失使用binary_csorssentropy;多分类,单标签使用softmax,损失函数用categorical_csorssentropy;多分类,多标签使用sigmoid,损失函数用binary_csorssentropy;回归到任意值,最后一层激活无,损失函数使用mse;回归到0-1的范围内的值,最后一层使用sigmoid,损失函数使用mse或binary_csorssentropy
  6. 扩大模型规模,开发过拟合的模型。添加更多的层,让每一层变得更大,训练更多的轮次.监控指标,如果性能开始下降,那么就出现了过拟合。
  7. 模型正则化和调整超参数,添加dropout,尝试不用的架构,添加L1或者L2正则化,尝试不同的超参数(每层的单元个数或优化器的学习率);反复做特征工程,添加新特征或者删除没有信息量的特征

简介

1
2
3
4
5
6
7
8
9
10
11
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))
model.add(layers.MaxPool2D((2,2)))
model.add(layers.Conv2D(64,(3,3),activation='relu'))
model.add(layers.MaxPool2D((2,2)))
model.add(layers.Conv2D(64,(3,3),activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64,activation='relu'))
model.add(layers.Dense(10,activation='softmax'))
model.summary()

image-20210412093459534

宽度和高度两个维度尺寸随着网络加深而变小,通道数由第一个参数控制(32或64).最后一层相当于10个分类。输出10个的softmax激活

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from keras.datasets import mnist
from keras.utils import to_categorical
(train_images,train_labels),(test_images,test_labels) = mnist.load_data()
train_images = train_images.reshape(60000,28,28,1)
train_images = train_images.astype('float32')/255

test_images = test_images.reshape(10000,28,28,1)
test_images = test_images.astype('float32')/255

train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)

model.compile(optimizer='rmsprop',
             loss='categorical_crossentropy',
             metrics=['accuracy'])
model.fit(train_images,train_labels,epochs=5,batch_size=64)
test_loss, test_acc = model.evaluate(test_images,test_labels)
print(test_acc) # 手写数字识别

image-20210412094940235

猫狗分类

分为训练,验证和测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import os, shutil
original_dataset_dir = '/Users/Yun/Desktop/wangweijie/data/natural_images'
base_dir = '/Users/Yun/Desktop/wangweijie/cats_and_dogs_small'
os.mkdir(base_dir)

# 构证训练集，验证集，测试集
train_dir = os.path.join(base_dir,'train')
os.mkdir(train_dir)
validation_dir = os.path.join(base_dir,'validations')
os.mkfifo(validation_dir)
test_dir = os.path.join(base_dir,'test')
os.mkdir(test_dir)

#构建猫狗的训练和验证与测试目录
train_cats_dir = os.path.join(train_dir,'cats')
os.mkdir(train_cats_dir)
train_dogs_dir = os.path.join(train_dir,'dogs')
os.mkdir(train_dogs_dir)

validation_cats_dir = os.path.join(validation_dir,'cats')
os.mkdir(validation_cats_dir)
validation_dog_dir = os.path.join(validation_dir,'dogs')
os.mkdir(validation_dog_dir)


test_cats_dir = os.path.join(test_dir,'cats')
os.mkdir(test_cats_dire)
test_dogs_dir = os.path.join(test_dir,'dogs')
os.mkdir(test_dogs_dir)

训练集,验证集200,测试集为6:3:1

定义神经网络

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from keras import layers
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3)))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(64,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(128,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(128,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Flatten())
model.add(layers.Dense(512,activation='relu'))
model.add(layers.Dense(1,activation='sigmoid'))
model.summary()

配置模型

1
2
3
4
from keras import optimizers
model.compile(loss='binary_crossentropy',
             optimizer=optimizers.RMSprop(lr=1e-4),
             metrics=['acc'])

数据预处理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255) # 图像乘以1/255缩放

train_generator = train_datagen.flow_from_directory(
    '/Users/Yun/Desktop/wangweijie/cats_and_dogs_small/train',
    target_size=(150,150),
    batch_size=20,
    class_mode='binary'
)

validation_generator = test_datagen.flow_from_directory(
    '/Users/Yun/Desktop/wangweijie/cats_and_dogs_small/vali',
    target_size=(150,150),
    batch_size=20,
    class_mode='binary'
)

训练模型

1
2
3
4
5
6
history = model.fit_generator(
    train_generator,
    epochs=10,
    validation_data=validation_generator
    )
model.save('cats_and_dogs_small_1.h5')

衡量数据指标

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import matplotlib.pyplot as plt
acc = history.history['acc']
val_acc = history.history['val_acc']

loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1,len(acc)+1)

plt.plot(epochs,acc,'bo',label='t')
plt.plot(epochs,val_acc,'b',label='tv')
plt.legend()
plt.figure()

plt.plot(epochs,loss,'bo',label='tl')
plt.plot(epochs,val_loss,'b',label='tvl')
plt.legend()
plt.show()

image-20210412113451934

数据增强

使用dropout和数据增强器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
model = models.Sequential()
model.add(layers.Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3)))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(64,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(128,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Conv2D(128,(3,3),activation='relu'))
model.add(layers.MaxPool2D(2,2))
model.add(layers.Flatten())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(512,activation='relu'))
model.add(layers.Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',
             optimizer=optimizers.RMSprop(lr=1e-4),
             metrics=['acc'])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=40,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True
)

test_datagen = ImageDataGenerator(1./255)

train_generator = train_datagen.flow_from_directory(
    '/Users/Yun/Desktop/wangweijie/cats_and_dogs_small/train',
    target_size=(150,150),
    batch_size=20,
    class_mode='binary'
)

validation_generator = test_datagen.flow_from_directory(
    '/Users/Yun/Desktop/wangweijie/cats_and_dogs_small/vali',
    target_size=(150,150),
    batch_size=20,
    class_mode='binary'
)
1
2
3
4
5
6
7
history = model.fit(
    train_generator,
    epochs=10,
    validation_data=validation_generator
    )
model.save('/Users/Yun/Desktop/wangweijie/cats_and_dogs_small_2.h5')
# 因为数据集太少了。所以少数的训练次数还是不能够达到效果

image-20210412123535426