计算梯度

计算x^2的梯度

import numpyimport theanoimport theano.tensor as Tfrom theano import ppx = T.dscalar('x')y = x ** 2gy = T.grad(y, x)pp(gy)f = theano.function([x], gy)pp(f.maker.fgraph.outputs[0])f(4)numpy.allclose(f(94.2), 188.4)

计算逻辑函数的梯度

x = T.dmatrix('x')s = T.sum(1 / (1 + T.exp(-x)))gs = T.grad(s, x)dlogistic = theano.function([x], gs)dlogistic([[0, 1], [-1, -2]])

计算Jacobian

x = T.dvector('x')y = x ** 2J, updates = theano.scan(lambda i, y, x: T.grad(y[i], x), sequences=T.arange(y.shape[0]), non_sequences=[y,x])f = theano.function([x], J, updates=updates)f([4, 4])

计算Hessian矩阵

x = T.dvector('x')y = x ** 2cost = y.sum()gy = T.grad(cost, x)H, updates = theano.scan(lambda i, gy, x: T.grad(gy[i], x), sequences=T.arange(gy.shape[0]), non_sequences=[gy,x])f = theano.function([x], H, updates=updates)f([4,4])

Jacobian times a Vector

右算子(R-operator)

W = T.dmatrix('W')V = T.dmatrix('V')x = T.dvector('x')y = T.dot(x, W)JV = T.Rop(y, W, V)f = theano.function([W, V, x], JV)f([[1,1], [1,1]], [[2,2], [2,2]], [0,1])

左算子（L-operator)

W = T.dmatrix('W')v = T.dvector('v')x = T.dvector('x')y = T.dot(x, W)VJ = T.Lop(y, W, v)f = theano.function([v, x], VJ)f([2,2], [0,1])

Hessian times a Vector

x = T.dvector('x')v = T.dvector('v')y = T.sum(x ** 2)gy = T.grad(y, x)vH = T.grad(T.sum(gy * v), x)f = theano.function([x,v], vH)f([4,4], [2,2])

右算子

x = T.dvector('x')v = T.dvector('v')y = T.sum(x ** 2)gy = T.grad(y, x)Hv = T.Rop(gy, x, v)f = theano.function([x,v], Hv)f([4,4], [2,2])