International business machines corporation (20240119274). TRAINING NEURAL NETWORKS WITH CONVERGENCE TO A GLOBAL MINIMUM simplified abstract

TRAINING NEURAL NETWORKS WITH CONVERGENCE TO A GLOBAL MINIMUM

Organization Name

international business machines corporation

Inventor(s)

Lam Minh Nguyen of Ossining NY (US)

TRAINING NEURAL NETWORKS WITH CONVERGENCE TO A GLOBAL MINIMUM - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240119274 titled 'TRAINING NEURAL NETWORKS WITH CONVERGENCE TO A GLOBAL MINIMUM

Simplified Explanation

The patent application describes a method for training a neural network with a non-convex architecture by approximating a convex optimization sub-problem to learn a common classifier from training data. The method involves selecting an initial weight vector, approximating a solution to the sub-problem to obtain a search direction, updating the weight vector using a learning rate, and repeating the process until convergence to a global minimum is achieved.

• Select initial weight vector for a convex optimization sub-problem associated with a neural network.
• Approximate a solution to the convex optimization sub-problem to obtain a search direction.
• Update the initial weight vector by subtracting the approximate solution times a learning rate.
• Repeat the approximating and updating steps for multiple iterations until convergence to a global minimum is achieved.

Potential Applications

The technology can be applied in various fields such as image recognition, natural language processing, and autonomous driving.

Problems Solved

This technology addresses the challenge of training neural networks with non-convex architectures efficiently and effectively.

Benefits

The method allows for the training of neural networks with complex architectures by approximating convex optimization sub-problems, leading to improved performance and convergence to global minima.

Potential Commercial Applications

The technology can be utilized in industries such as healthcare, finance, and e-commerce for tasks like medical image analysis, fraud detection, and recommendation systems.

Possible Prior Art

Prior art may include techniques for training neural networks with non-convex architectures using different optimization algorithms or approaches.

How does this method compare to existing techniques for training neural networks with non-convex architectures?

The article does not provide a comparison with existing techniques, leaving the reader wondering about the advantages and limitations of this approach.

What are the specific parameters and hyperparameters used in the training process?

The article does not detail the specific parameters and hyperparameters used in the training process, which could impact the performance and efficiency of the method.

Original Abstract Submitted

select an initial weight vector for a convex optimization sub-problem associated with a neural network having a non-convex network architecture loss surface. with at least one processor, approximate a solution to the convex optimization sub-problem that obtains a search direction, to learn a common classifier from training data. with the at least one processor, update the initial weight vector by subtracting the approximate solution to the convex optimization sub-problem times a first learning rate. with the at least one processor, repeat the approximating and updating steps, for a plurality of iterations, with the updated weight vector from a given one of the iterations taken as the initial weight vector for a next one of the iterations, to obtain a final weight vector for the neural network, until convergence to a global minimum is achieved, to implement the common classifier.