keras如何打印loss对权重的导数

2024-12-13 技术教程

这篇文章主要讲解了keras如何打印loss对权重的导数，内容清晰明了，对此有兴趣的小伙伴可以学习一下，相信大家阅读完之后会有帮助。

Notes

怀疑模型梯度爆炸，想打印模型 loss 对各权重的导数看看。如果如果fit来训练的话，可以用keras.callbacks.TensorBoard实现。

但此次使用train_on_batch来训练的，用K.gradients和K.function实现。

Codes

以一份 VAE 代码为例

# -*- coding: utf8 -*-import kerasfrom keras.models import Modelfrom keras.layers import Input, Lambda, Conv2D, MaxPooling2D, Flatten, Dense, Reshapefrom keras.losses import binary_crossentropyfrom keras.datasets import mnist, fashion_mnistimport keras.backend as Kfrom scipy.stats import normimport numpy as npimport matplotlib.pyplot as pltBATCH = 128N_CLASS = 10EPOCH = 5IN_DIM = 28 * 28H_DIM = 128Z_DIM = 2(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()x_train = x_train.reshape(len(x_train), -1).astype('float32') / 255.x_test = x_test.reshape(len(x_test), -1).astype('float32') / 255.def sampleing(args): """reparameterize""" mu, logvar = args eps = K.random_normal([K.shape(mu)[0], Z_DIM], mean=0.0, stddev=1.0) return mu + eps * K.exp(logvar / 2.)# encodex_in = Input([IN_DIM])h = Dense(H_DIM, activation='relu')(x_in)z_mu = Dense(Z_DIM)(h) # mean，不用激活z_logvar = Dense(Z_DIM)(h) # log variance，不用激活z = Lambda(sampleing, output_shape=[Z_DIM])([z_mu, z_logvar]) # 只能有一个参数encoder = Model(x_in, [z_mu, z_logvar, z], name='encoder')# decodez_in = Input([Z_DIM])h_hat = Dense(H_DIM, activation='relu')(z_in)x_hat = Dense(IN_DIM, activation='sigmoid')(h_hat)decoder = Model(z_in, x_hat, name='decoder')# VAEx_in = Input([IN_DIM])x = x_inz_mu, z_logvar, z = encoder(x)x = decoder(z)out = xvae = Model(x_in, [out, out], name='vae')# loss_kl = 0.5 * K.sum(K.square(z_mu) + K.exp(z_logvar) - 1. - z_logvar, axis=1)# loss_recon = binary_crossentropy(K.reshape(vae_in, [-1, IN_DIM]), vae_out) * IN_DIM# loss_vae = K.mean(loss_kl + loss_recon)def loss_kl(y_true, y_pred): return 0.5 * K.sum(K.square(z_mu) + K.exp(z_logvar) - 1. - z_logvar, axis=1)# vae.add_loss(loss_vae)vae.compile(optimizer='rmsprop', loss=[loss_kl, 'binary_crossentropy'], loss_weights=[1, IN_DIM])vae.summary()# 获取模型权重 variablew = vae.trainable_weightsprint(w)# 打印 KL 对权重的导数# KL 要是 Tensor，不能是上面的函数 `loss_kl`grad = K.gradients(0.5 * K.sum(K.square(z_mu) + K.exp(z_logvar) - 1. - z_logvar, axis=1), w)print(grad) # 有些是 None 的grad = grad[grad is not None] # 去掉 None，不然报错# 打印梯度的函数# K.function 的输入和输出必要是 list！就算只有一个show_grad = K.function([vae.input], [grad])# vae.fit(x_train, # y_train, # 不能传 y_train# batch_size=BATCH,# epochs=EPOCH,# verbose=1,# validation_data=(x_test, None))''' 以 train_on_batch 方式训练 '''for epoch in range(EPOCH): for b in range(x_train.shape[0] // BATCH): idx = np.random.choice(x_train.shape[0], BATCH) x = x_train[idx] l = vae.train_on_batch([x], [x, x]) # 计算梯度 gd = show_grad([x]) # 打印梯度 print(gd)# show manifoldPIXEL = 28N_PICT = 30grid_x = norm.ppf(np.linspace(0.05, 0.95, N_PICT))grid_y = grid_xfigure = np.zeros([N_PICT * PIXEL, N_PICT * PIXEL])for i, xi in enumerate(grid_x): for j, yj in enumerate(grid_y): noise = np.array([[xi, yj]]) # 必须秩为 2，两层中括号 x_gen = decoder.predict(noise) # print('x_gen shape:', x_gen.shape) x_gen = x_gen[0].reshape([PIXEL, PIXEL]) figure[i * PIXEL: (i+1) * PIXEL, j * PIXEL: (j+1) * PIXEL] = x_genfig = plt.figure(figsize=(10, 10))plt.imshow(figure, cmap='Greys_r')fig.savefig('./variational_autoencoder.png')plt.show()

补充知识：keras 自定义损失自动求导时出现None

问题记录，keras 自定义损失自动求导时出现None，后来想到是因为传入的变量没有使用，所以keras无法求出偏导，修改后问题解决。就是不愿使用的变量×0,求导后还是0就可以了。

def my_complex_loss_graph(y_label, emb_uid, lstm_out,y_true_1,y_true_2,y_true_3,out_1,out_2,out_3): mse_out_1 = mean_squared_error(y_true_1, out_1) mse_out_2 = mean_squared_error(y_true_2, out_2) mse_out_3 = mean_squared_error(y_true_3, out_3) # emb_uid= K.reshape(emb_uid, [-1, 32]) cosine_sim = tf.reduce_sum(0.5*tf.square(emb_uid-lstm_out)) cost=0*cosine_sim+K.sum([0.5*mse_out_1 , 0.25*mse_out_2,0.25*mse_out_3],axis=1,keepdims=True) # print(mse_out_1) final_loss = cost return K.mean(final_loss)

看完上述内容，是不是对keras如何打印loss对权重的导数有进一步的了解，如果还想学习更多内容，欢迎关注亿速云行业资讯频道。