TensorFlow-SlimTF?Slim 是 2016 年開源庫,，主要用于"代碼瘦身",，便于模型定義,，并給出了一些圖像分析模型. TF-Slim 是用于 TensorFlow 復(fù)雜模型的定義、訓(xùn)練和評估的輕量庫. [tensorflow/contrib/slim] 模塊導(dǎo)入： import tensorflow.contrib.slim as slim <h2>1. TF-Slim 特點</h2> TF-Slim 用于神經(jīng)網(wǎng)絡(luò)的構(gòu)建,、訓(xùn)練和評估：
<h2>2. TF-Slim 組成</h2> TF-Slim 由獨立的幾個部分組成，主要有：
<h2>3. TF-Slim 定義模型</h2> TF-Slim 通過結(jié)合 variables, layers 和 scopes 進行模型定義. <h3>3.1 Variables</h3> TensorFlow 的原始變量 Variables 定義需要預(yù)定義值或者初始化方法如,，隨機采樣如,，Gaissian隨機采樣. TF-Slim 提供了更輕量的變量封裝函數(shù) - variables.py. 例如,，創(chuàng)建 weights = slim.variable('weights',
shape=[10, 10, 3 , 3],
initializer=tf.truncated_normal_initializer(stddev=0.1),
regularizer=slim.l2_regularizer(0.05),
device='/CPU:0')TensorFlow 原始實現(xiàn)中,，有兩種類型的變量：regular variables 和 localtransientvariables. TF-Slim 通過定義模型變量，進一步對變量區(qū)分. 模型變量表示了模型的參數(shù)modelvariables. 非模型變量non?modelvariables是網(wǎng)絡(luò)學(xué)習(xí)或評估時時的所有其它模型變量,，但真實推斷不需要的變量. 采用 TF-Slim 可以很簡單的創(chuàng)建與檢索模型變量modelvariables和正則變量regularvariables： # Model Variablesweights = slim.model_variable('weights',
shape=[10, 10, 3 , 3],
initializer=tf.truncated_normal_initializer(stddev=0.1),
regularizer=slim.l2_regularizer(0.05),
device='/CPU:0')
model_variables = slim.get_model_variables()# Regular variablesmy_var = slim.variable('my_var',
shape=[20, 1],
initializer=tf.zeros_initializer())
regular_variables_and_model_variables = slim.get_variables()這是怎么實現(xiàn)的呢,？ 如果需要自定義網(wǎng)絡(luò)層或變量創(chuàng)建方法,，仍想 TF-Slim 來管理模型變量呢？ my_model_variable = CreateViaCustomCode()# Letting TF-Slim know about the additional variable.slim.add_model_variable(my_model_variable) <h3>3.2 Layers</h3> TensorFlow Ops 是非常廣泛的,，神經(jīng)網(wǎng)絡(luò)開發(fā)者對于模型是比較高層的概念，如 Layers,，Losses,， Metrics，Networks. 網(wǎng)絡(luò)層Layer,，如卷積層ConvLayer,，全連接層FCLayer，BatchNormLayer,，比 TensorFlow Ops 更抽象,，且一般涉及多個 Ops.
ConvLayer 基于原始 TensorFlow 實現(xiàn),，相當(dāng)繁瑣： input = ...with tf.name_scope('conv1_1') as scope:
kernel = tf.Variable(tf.truncated_normal([3, 3, 64, 128], dtype=tf.float32,
stddev=1e-1), name='weights')
conv = tf.nn.conv2d(input, kernel, [1, 1, 1, 1], padding='SAME')
biases = tf.Variable(tf.constant(0.0, shape=[128], dtype=tf.float32),
trainable=True, name='biases')
bias = tf.nn.bias_add(conv, biases)
conv1 = tf.nn.relu(bias, name=scope)為了減少代碼的重復(fù),，TF-Slim 提供了很多方便的 Ops，更抽象的定義網(wǎng)絡(luò)層. input = ... net = slim.conv2d(input, 128, [3, 3], scope='conv1_1') TF-Slim 提供了很多網(wǎng)絡(luò)構(gòu)建的標(biāo)準(zhǔn)網(wǎng)絡(luò)層：
TF-Slim 還提供了兩個 meta-operations： net = ... net = slim.conv2d(net, 256, [3, 3], scope='conv3_1') net = slim.conv2d(net, 256, [3, 3], scope='conv3_2') net = slim.conv2d(net, 256, [3, 3], scope='conv3_3') net = slim.max_pool2d(net, [2, 2], scope='pool2') 采用 net = ...for i in range(3): net = slim.conv2d(net, 256, [3, 3], scope='conv3_%d' % (i+1)) net = slim.max_pool2d(net, [2, 2], scope='pool2') 采用 TF-Slim 的 net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3') net = slim.max_pool2d(net, [2, 2], scope='pool2')
TF-Slim 的 # 冗長方式:x = slim.fully_connected(x, 32, scope='fc/fc_1') x = slim.fully_connected(x, 64, scope='fc/fc_2') x = slim.fully_connected(x, 128, scope='fc/fc_3')# 等價方式, TF-Slim way using slim.stack:slim.stack(x, slim.fully_connected, [32, 64, 128], scope='fc') 該例中， 類似地,，多卷積層的堆積： # Verbose way:x = slim.conv2d(x, 32, [3, 3], scope='core/core_1') x = slim.conv2d(x, 32, [1, 1], scope='core/core_2') x = slim.conv2d(x, 64, [3, 3], scope='core/core_3') x = slim.conv2d(x, 64, [1, 1], scope='core/core_4')# Using stack:slim.stack(x, slim.conv2d, [(32, [3, 3]), (32, [1, 1]), (64, [3, 3]), (64, [1, 1])], scope='core') <h3>3.3 Scopes</h3> 除了 TensorFlow 中的作用域類型scopetype- name_scope 和 variable_scope，TF-Slim 新增了一個作用域類型 - arg_scope. arg_scope 用于指定一個或多個 Ops,，以及指定傳遞到在 例如： net = slim.conv2d(inputs, 64, [11, 11], 4, padding='SAME', weights_initializer=tf.truncated_normal_initializer(stddev=0.01), weights_regularizer=slim.l2_regularizer(0.0005), scope='conv1') net = slim.conv2d(net, 128, [11, 11], padding='VALID', weights_initializer=tf.truncated_normal_initializer(stddev=0.01), weights_regularizer=slim.l2_regularizer(0.0005), scope='conv2') net = slim.conv2d(net, 256, [11, 11], padding='SAME', weights_initializer=tf.truncated_normal_initializer(stddev=0.01), weights_regularizer=slim.l2_regularizer(0.0005), scope='conv3') 三個 ConvLayers 共享相同的超參數(shù),，其中兩個具有相同的 padding，三個都是相同的 weights_initializer 和 weight_regularizer. padding = 'SAME'initializer = tf.truncated_normal_initializer(stddev=0.01) regularizer = slim.l2_regularizer(0.0005) net = slim.conv2d(inputs, 64, [11, 11], 4, padding=padding, weights_initializer=initializer, weights_regularizer=regularizer, scope='conv1') net = slim.conv2d(net, 128, [11, 11], padding='VALID', weights_initializer=initializer, weights_regularizer=regularizer, scope='conv2') net = slim.conv2d(net, 256, [11, 11], padding=padding, weights_initializer=initializer, weights_regularizer=regularizer, scope='conv3') 該方式可以保證三個 ConvLayer 共享相同的參數(shù),，但并沒有完全減少代碼冗余. 采用 with slim.arg_scope([slim.conv2d], padding='SAME', weights_initializer=tf.truncated_normal_initializer(stddev=0.01) weights_regularizer=slim.l2_regularizer(0.0005)): net = slim.conv2d(inputs, 64, [11, 11], scope='conv1') net = slim.conv2d(net, 128, [11, 11], padding='VALID', scope='conv2') net = slim.conv2d(net, 256, [11, 11], scope='conv3')
另外,，也可以內(nèi)嵌 with slim.arg_scope([slim.conv2d, slim.fully_connected], activation_fn=tf.nn.relu, weights_initializer=tf.truncated_normal_initializer(stddev=0.01), weights_regularizer=slim.l2_regularizer(0.0005)):with slim.arg_scope([slim.conv2d], stride=1, padding='SAME'): net = slim.conv2d(inputs, 64, [11, 11], 4, padding='VALID', scope='conv1') net = slim.conv2d(net, 256, [5, 5], weights_initializer=tf.truncated_normal_initializer(stddev=0.03), scope='conv2') net = slim.fully_connected(net, 1000, activation_fn=None, scope='fc') 該例中,，第一個 <h3>3.4 VGG16 網(wǎng)絡(luò)層示例</h3> 利用 TF-Slim 的 Variables，Operation 和 Scopes,，創(chuàng)建 VGG16 網(wǎng)絡(luò)： def vgg16(inputs): with slim.arg_scope([slim.conv2d, slim.fully_connected], activation_fn=tf.nn.relu, weights_initializer=tf.truncated_normal_initializer(0.0, 0.01), weights_regularizer=slim.l2_regularizer(0.0005)): net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1') net = slim.max_pool2d(net, [2, 2], scope='pool1') net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2') net = slim.max_pool2d(net, [2, 2], scope='pool2') net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3') net = slim.max_pool2d(net, [2, 2], scope='pool3') net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4') net = slim.max_pool2d(net, [2, 2], scope='pool4') net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5') net = slim.max_pool2d(net, [2, 2], scope='pool5') net = slim.fully_connected(net, 4096, scope='fc6') net = slim.dropout(net, 0.5, scope='dropout6') net = slim.fully_connected(net, 4096, scope='fc7') net = slim.dropout(net, 0.5, scope='dropout7') net = slim.fully_connected(net, 1000, activation_fn=None, scope='fc8') return net <h2>4. 模型訓(xùn)練</h2> TensorFlow 模型訓(xùn)練需要模型Model,，Loss 函數(shù)，梯度計算和迭代的計算模型權(quán)重相對于 loss 的梯度和對應(yīng)的權(quán)重更新的訓(xùn)練方案. TF-Slim 提供了常用 loss 函數(shù)和 helper 函數(shù),，以進行模型訓(xùn)練和評估. <h3>4.1 Losses</h3> Loss 函數(shù)定義了需要最小化的目標(biāo). 某些模型,，如 multi-task learning 模型,，需要同時采用多個 loss. 也就是說，最終的 loss 函數(shù)是不同 loss 函數(shù)之和的最小化. TF-Slim 提供了一種易用的 loss 函數(shù)定義機制，采用了 losses 模塊. import tensorflow as tfimport tensorflow.contrib.slim.nets as nets vgg = nets.vgg# Load the images and labels.images, labels = ...# Create the model.predictions, _ = vgg.vgg_16(images)# Define the loss functions and get the total loss.loss = slim.losses.softmax_cross_entropy(predictions, labels) 該例中,，先創(chuàng)建模型采用的實現(xiàn)采用TF?Slim的VGG實現(xiàn),，然后添加標(biāo)準(zhǔn)的分類 loss. 現(xiàn)在,，針對 multi-task 模型的情況，模型有多個輸出. # Load the images and labels.images, scene_labels, depth_labels = ...# Create the model.scene_predictions, depth_predictions = CreateMultiTaskModel(images)# Define the loss functions and get the total loss.classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels) sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels)# The following two lines have the same effect:total_loss = classification_loss + sum_of_squares_loss total_loss = slim.losses.get_total_loss(add_regularization_losses=False) 該例中,，有兩個 losses： 如果自定義 loss 函數(shù),，如何采用 TF-Slim 來管理 losses 呢？ # Load the images and labels.images, scene_labels, depth_labels, pose_labels = ...# Create the model.scene_predictions, depth_predictions, pose_predictions = CreateMultiTaskModel(images)# Define the loss functions and get the total loss.classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels) sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels) pose_loss = MyCustomLossFunction(pose_predictions, pose_labels) slim.losses.add_loss(pose_loss) # Letting TF-Slim know about the additional loss.# The following two ways to compute the total loss are equivalent:regularization_loss = tf.add_n(slim.losses.get_regularization_losses()) total_loss1 = classification_loss + sum_of_squares_loss + pose_loss + regularization_loss# (Regularization Loss is included in the total loss by default).total_loss2 = slim.losses.get_total_loss() <h3>4.2 訓(xùn)練循環(huán)</h3> TF-Slim 提供了簡單有效的模型訓(xùn)練工具 - learning.py. 例如,，當(dāng)模型、loss 函數(shù)和優(yōu)化策略定義完成后,，即可調(diào)用 g = tf.Graph()# Create the model and specify the losses...... total_loss = slim.losses.get_total_loss() optimizer = tf.train.GradientDescentOptimizer(learning_rate)# create_train_op ensures that each time we ask for the loss, the update_ops# are run and the gradients being computed are applied too.train_op = slim.learning.create_train_op(total_loss, optimizer) logdir = ... # Where checkpoints are stored.slim.learning.train( train_op, logdir, number_of_steps=1000, save_summaries_secs=300, save_interval_secs=600): 該例中,，
<h3>4.3 VGG 模型訓(xùn)練示例</h3> import tensorflow as tfimport tensorflow.contrib.slim.nets as nets
slim = tf.contrib.slim
vgg = nets.vgg
...
train_log_dir = ...if not tf.gfile.Exists(train_log_dir):
tf.gfile.MakeDirs(train_log_dir)with tf.Graph().as_default(): # Set up the data loading:
images, labels = ... # Define the model:
predictions = vgg.vgg_16(images, is_training=True) # Specify the loss function:
slim.losses.softmax_cross_entropy(predictions, labels)
total_loss = slim.losses.get_total_loss()
tf.summary.scalar('losses/total_loss', total_loss) # Specify the optimization scheme:
optimizer = tf.train.GradientDescentOptimizer(learning_rate=.001) # create_train_op that ensures that when we evaluate it to get the loss,
# the update_ops are done and the gradient updates are computed.
train_tensor = slim.learning.create_train_op(total_loss, optimizer) # Actually runs training.
slim.learning.train(train_tensor, train_log_dir)<h2>5. 模型 fine-tuning</h2> <h3>5.1 簡單回顧 - 從斷點文件恢復(fù)模型變量</h3> 模型訓(xùn)練后,，可以采用 # Create some variables.v1 = tf.Variable(..., name="v1")
v2 = tf.Variable(..., name="v2")
...# Add ops to restore all the variables.restorer = tf.train.Saver()# Add ops to restore some variables.restorer = tf.train.Saver([v1, v2])# Later, launch the model, use the saver to restore variables from disk, and# do some work with the model.with tf.Session() as sess:# Restore variables from disk.restorer.restore(sess, "/tmp/model.ckpt")
print("Model restored.")# Do some work with the model...更多細(xì)節(jié)可以參考：Restoring Variables 和 Choosing which Variables to Save and Restore <h3>5.2 部分恢復(fù)模型</h3> 在新的數(shù)據(jù)集和新任務(wù)的情況下,，往往需要采用在 pre-trained 模型上 fine-tune. # Create some variables.v1 = slim.variable(name="v1", ...)
v2 = slim.variable(name="nested/v2", ...)
...# Get list of variables to restore (which contains only 'v2'). These are all# equivalent methods:variables_to_restore = slim.get_variables_by_name("v2")# orvariables_to_restore = slim.get_variables_by_suffix("2")# orvariables_to_restore = slim.get_variables(scope="nested")# orvariables_to_restore = slim.get_variables_to_restore(include=["nested"])# orvariables_to_restore = slim.get_variables_to_restore(exclude=["v1"])# Create the saver which will be used to restore the variables.restorer = tf.train.Saver(variables_to_restore)with tf.Session() as sess:# Restore variables from disk.restorer.restore(sess, "/tmp/model.ckpt")
print("Model restored.")# Do some work with the model...<h3>5.3 恢復(fù)不同變量名的模型</h3> 當(dāng)從斷點文件恢復(fù)變量時,， 上面中,，創(chuàng)建 saver 來傳遞變量. 這里,，從每個提供的變量的 例如： # Assuming than 'conv1/weights' should be restored from 'vgg16/conv1/weights'def name_in_checkpoint(var):return 'vgg16/' + var.op.name# Assuming than 'conv1/weights' and 'conv1/bias' should be restored from 'conv1/params1' and 'conv1/params2'def name_in_checkpoint(var):if "weights" in var.op.name:return var.op.name.replace("weights", "params1")if "bias" in var.op.name:return var.op.name.replace("bias", "params2")
variables_to_restore = slim.get_model_variables()
variables_to_restore = {name_in_checkpoint(var):var for var in variables_to_restore}
restorer = tf.train.Saver(variables_to_restore)with tf.Session() as sess: # Restore variables from disk.
restorer.restore(sess, "/tmp/model.ckpt")<h3>5.4 在不同任務(wù) Fine-tuning 模型</h3> 假如，已經(jīng)有預(yù)訓(xùn)練的 VGG16 模型,，其是在 ImageNet 數(shù)據(jù)集上訓(xùn)練得到,，1000 類的分類模型. 此種情況,，可以采用預(yù)訓(xùn)練模型初始化模型訓(xùn)練,，但排除最后一網(wǎng)絡(luò)層： # Load the Pascal VOC dataimage, label = MyPascalVocDataLoader(...) images, labels = tf.train.batch([image, label], batch_size=32)# Create the modelpredictions = vgg.vgg_16(images) train_op = slim.learning.create_train_op(...)# Specify where the Model, trained on ImageNet, was saved.model_path = '/path/to/pre_trained_on_imagenet.checkpoint'# Specify where the new model will live:log_dir = '/path/to/my_pascal_model_dir/'# Restore only the convolutional layers:variables_to_restore = slim.get_variables_to_restore(exclude=['fc6', 'fc7', 'fc8']) init_fn = assign_from_checkpoint_fn(model_path, variables_to_restore)# Start training.slim.learning.train(train_op, log_dir, init_fn=init_fn) <h2>6. 模型評估</h2> 當(dāng)模型訓(xùn)練后，往往需要評估模型的實際表現(xiàn). <h3>6.1 Metrics</h3> 定義 metric 來評估模型表現(xiàn),，但不是 loss 函數(shù)一般是在訓(xùn)練時直接優(yōu)化loss一般是在訓(xùn)練時直接優(yōu)化. TF-Slim 提供了很多 metric Ops,，以易于評估模型.
例如，為了計算 例如： images, labels = LoadTestData(...) predictions = MyModel(images) mae_value_op, mae_update_op = slim.metrics.streaming_mean_absolute_error(predictions, labels) mre_value_op, mre_update_op = slim.metrics.streaming_mean_relative_error(predictions, labels) pl_value_op, pl_update_op = slim.metrics.percentage_less(mean_relative_errors, 0.3) TF-Slim 還提供了兩個函數(shù)： # Aggregates the value and update ops in two lists:value_ops, update_ops = slim.metrics.aggregate_metrics(
slim.metrics.streaming_mean_absolute_error(predictions, labels),
slim.metrics.streaming_mean_squared_error(predictions, labels))# Aggregates the value and update ops in two dictionaries:names_to_values, names_to_updates = slim.metrics.aggregate_metric_map({"eval/mean_absolute_error": slim.metrics.streaming_mean_absolute_error(predictions, labels),"eval/mean_squared_error": slim.metrics.streaming_mean_squared_error(predictions, labels),
})<h3>6.2 追蹤多個 Metrics 示例</h3> import tensorflow as tfimport tensorflow.contrib.slim.nets as nets
slim = tf.contrib.slim
vgg = nets.vgg# 加載數(shù)據(jù)images, labels = load_data(...)# 定義網(wǎng)絡(luò)predictions = vgg.vgg_16(images)# 選擇計算的 metrics:names_to_values, names_to_updates = slim.metrics.aggregate_metric_map({"eval/mean_absolute_error": slim.metrics.streaming_mean_absolute_error(predictions, labels),"eval/mean_squared_error": slim.metrics.streaming_mean_squared_error(predictions, labels),
})# Evaluate the model using 1000 batches of data:num_batches = 1000with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())for batch_id in range(num_batches):
sess.run(names_to_updates.values())
metric_values = sess.run(names_to_values.values())for metric, value in zip(names_to_values.keys(), metric_values):
print('Metric %s has value: %f' % (metric, value))metric.py 可以在沒有 layers 或 loss_ops.py 時獨立使用. <h3>6.3 循環(huán)評估</h3> TF-Slim 提供了評估模塊 - evaluation.py，包含了采用 metrics 從metric_ops.py 寫入評測腳本的 helper 函數(shù). 主要包括,，周期地運行評估,，對 batch 數(shù)據(jù)計算 metric，以及打印和 summarizing metric 結(jié)果. 例如： import tensorflow as tf
slim = tf.contrib.slim# Load the dataimages, labels = load_data(...)# Define the networkpredictions = MyModel(images)# Choose the metrics to compute:names_to_values, names_to_updates = slim.metrics.aggregate_metric_map({'accuracy': slim.metrics.accuracy(predictions, labels),'precision': slim.metrics.precision(predictions, labels),'recall': slim.metrics.recall(mean_relative_errors, 0.3),
})# Create the summary ops such that they also print out to std output:summary_ops = []for metric_name, metric_value in names_to_values.iteritems():
op = tf.summary.scalar(metric_name, metric_value)
op = tf.Print(op, [metric_value], metric_name)
summary_ops.append(op)
num_examples = 10000batch_size = 32num_batches = math.ceil(num_examples / float(batch_size))# Setup the global step.slim.get_or_create_global_step()
output_dir = ... # Where the summaries are stored.eval_interval_secs = ... # How often to run the evaluation.slim.evaluation.evaluation_loop('local',
checkpoint_dir,
log_dir,
num_evals=num_batches,
eval_op=names_to_updates.values(),
summary_op=tf.summary.merge(summary_ops),
eval_interval_secs=eval_interval_secs)
|
|
|
