SimonBrain
Search
CTRL + K
SimonBrain
Search
CTRL + K
1-projects
parallel-hypergraph-neural-networks
hypergraph-definition
hypergraph-neural-networks
3-ressources
machine-learning
activation-functions
evidence-lower-bound
expectation-maximization-algorightm
gaussian-mixture-models
glove-model
gradient-descent
k-means-model
maximum-likelihood-estimation
mixture-models
non-negative-matrix-factorization
pooling-layer
probabilistic-latent-semantic-analysis
reparametrization-trick
math
linear-algebra
norms
frobenius-norm
nuclear-norm
characteristic-polynomial
cil-randomized-algorithm-svd
cil-reconstruction-theorem
cil-shrink-operator
covariance-matrix
dimension-reduction
eigen-decomposition
eigenvalue
eigenvector
fenchel-conjugate
hadamard-product
nuclear-norm-minimization
singular-value-decomposition
singular-value
variance-covariance-matrix
convexity
frobenius-norm
jensen-inequality
l-smoothness
lagrange-multipliers
4-archive
cil
project
cil-results
cil-trained-models
theory
old
01-00-dimension-reduction
02-00-matrix-approximation
02-01-recommender-systems
02-02-collaborative-filtering
02-03-netflix-data
02-04-preprocessing-normalization
02-05-rank-one-model
02-06-convextiy-definition
02-07-gradients
02-08-gradient-dynamics
02-09-fully-observed-rank-one-model
03-00-singular-value-decomposition
03-01-eckart-young-theorem
03-02-svd-pca
03-03-svd-matrix-completion
03-04-svd-with-imputation
03-05-np-hard
cil-important-points
cil-theory-overview
home
#machine-learning, eth/cil/theory
Gradient Descent
Workhorse of Deep Learning
most used method for learning
very generic and scalable
Definition Update Step
a
n
+
1
=
a
n
−
γ
∇
F
(
a
n
)
Statistical Gradient Descent
picks data point at random and evolves the iterates