skill-050

Requirements for Outputs

General Guidelines

Optimization

• All optimization routines MUST support JAX-compatible parameters and gradients.
• Ensure numerical stability and convergence for all optimization methods.
• Provide informative error messages for invalid inputs or configurations.

JAX Optimization Techniques

1. Gradient Descent

gradient_descent(fn, init_params, learning_rate, num_steps)

Description: Perform gradient descent optimization on a loss function.
Parameters:

• fn (callable): The loss function to minimize.
• init_params (array): Initial parameters for optimization.
• learning_rate (float): Step size for each iteration.
• num_steps (int): Number of optimization steps.

Returns: Optimized parameters after the specified number of steps.

import jax
import jax.numpy as jnp
from jax import grad
import jax_optimization_techniques as jx

# Define a simple quadratic loss function
def loss_fn(params):
    return jnp.sum((params - 3) ** 2)

# Perform optimization
optimized_params = jx.gradient_descent(loss_fn, jnp.array([0.0]), learning_rate=0.1, num_steps=100)

2. Adam Optimizer

adam_optimizer(fn, init_params, learning_rate, num_steps)

Description: Optimize a loss function using the Adam optimization algorithm.
Parameters:

• fn (callable): The loss function to minimize.
• init_params (array): Initial parameters for optimization.
• learning_rate (float): Step size for each iteration.
• num_steps (int): Number of optimization steps.

Returns: Optimized parameters after the specified number of steps.

# Initialize Adam optimizer
def adam_optimizer(fn, init_params, learning_rate=0.001, num_steps=100):
    params = init_params
    m = jax.numpy.zeros_like(params)
    v = jax.numpy.zeros_like(params)
    beta1 = 0.9
    beta2 = 0.999
    epsilon = 1e-8

    for t in range(1, num_steps + 1):
        g = grad(fn)(params)
        m = beta1 * m + (1 - beta1) * g
        v = beta2 * v + (1 - beta2) * (g ** 2)
        m_hat = m / (1 - beta1 ** t)
        v_hat = v / (1 - beta2 ** t)
        params -= learning_rate * m_hat / (jnp.sqrt(v_hat) + epsilon)
    return params

optimized_params = adam_optimizer(loss_fn, jnp.array([0.0]), learning_rate=0.01, num_steps=100)