Test Post

Mucong Ding

Mucong Ding / October 18, 2022

1 min read––– views

Test title

title does not support dark mode now

Smaller title

Much smaller title

smaller titles are OK

  1. Test list
  2. TODO
    • even more
      1. nested
  • rich markdown

    • wjisdf

$$math mode is not supported now$$

how to adjust image size?

def hypergradient(validation_loss, training_loss, lambda_, w:):
    v1 = torch.autograd.grad(validation_loss, w(), retain_graph=True)
    d_train_d_w = torch.autograd.grad(training_loss, w(), create_graph=True)
    v2 = approxInverseHVP(v1, d_train_d_w, w)
    v3 = torch.autograd.grad(d_train_d_w, lambda_(), grad_outputs=v2, retain_graph=True, )
    d_val_d_lambda = torch.autograd.grad(validation_loss, lambda_())
    return [d - v for d, v in zip(d_val_d_lambda, v3)]

def approxInverseHVP(v, f, w, i=3, alpha=.1):
    p = v
    for j in range(i):
        grad = torch.autograd.grad(f, w(), grad_outputs=v, retain_graph=True)
        v = [v_ - alpha * g for v_, g in zip(v, grad)]
        p = [p_ + v_ for p_, v_ in zip(p, v)]
    return p

code style looks OK but much to be prettified

Subscribe to the newsletter

Get emails from me about web development, tech, and early access to new articles.

- subscribers – View all issues