Patent Application 17654225 - COMPUTER SYSTEM AND METHOD FOR UTILIZING

Title: COMPUTER SYSTEM AND METHOD FOR UTILIZING VARIATIONAL INFERENCE

Application Information

Invention Title: COMPUTER SYSTEM AND METHOD FOR UTILIZING VARIATIONAL INFERENCE
Application Number: 17654225
Submission Date: 2025-05-16T00:00:00.000Z
Effective Filing Date: 2022-03-09T00:00:00.000Z
Filing Date: 2022-03-09T00:00:00.000Z
National Class: 706
National Sub-Class: 045000
Examiner Employee Number: 99681
Art Unit: 2129
Tech Center: 2100

Rejection Summary

102 Rejections: 0
103 Rejections: 4

Cited Patents

No patents were cited in this rejection.

Office Action Text

DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .

Drawings
The drawings are objected to because Fig. 12 element 160 should read as "Real physical system" instead of "Real physical systen". Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.

Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.

The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.

Claim 19 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 19 recites “specialized variational inference functions.” The expression "variational inference function" does not have a generally accepted specific technical meaning in the field of the instant application. In the specification of the instant application, the only recitation of a specialized… function is in para. [0120] recites “Beneficially, by using nested models, each model can become specialized at performing its specialized function, so the models in synergy provide a superlative vehicle control and driving accuracy. Such a manner of configuring the nested series of models is application to other use applications, for example aircraft navigation and flight control, rocket flight control, ship navigation, robotics, manufacturing machinery control, chemical processing works control, and so forth.” There is no description of “variational inference functions” in the specification and thus the recitation of “specialized variational inference functions” fails to comply with the written description requirement.

Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.

The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.

Claim 21, 29, 33 and 35 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21 and analogous claims 29, 33 and 35 are rejected as being indefinite as the claim recites “minimizing a Kullback-Leibler (KL) divergence of a true posterior and relying on a classifier that estimated a probability ratio;” KL divergence is a measure for indicating the closeness of two probability densities, it is unclear what the second probability density is. Further, it is unclear to what extent the KL divergence relies on a classifier to estimate a probability ratio and whether the probability ratio is a ratio of the first and second probabilities from the KL divergence or a separate probability ratio of two distinct probabilities.

Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.

The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 15-17, 23-25, 31-32, and 34 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Pub. No. US20180232649A1 Wiebe et al. (“Wiebe”) in view of Benedetti, Marcello, et al. "Adversarial quantum circuit learning for pure state approximation." New Journal of Physics 21.4 (2019): 043023. (“Benedetti”)
In regards to claim 15,
Wiebe teaches A control system for controlling or monitoring a real physical system, wherein the control system comprises a hybrid combination of a classical computer and a quantum computer,

PNG
media_image1.png
605
601
media_image1.png
Greyscale
(Wiebe, “[0073] With reference to FIG. 10, an exemplary system [A control system] for implementing some aspects of the disclosed technology includes a computing environment 1000 that includes a quantum processing unit 1002 and one or more monitoring/measuring device(s) 1046 [monitoring a real physical system]. The quantum processor executes quantum circuits (such as the circuit of FIG. 2) that are precompiled by classical compiler unit 1020 utilizing one or more classical processor(s) 1010 [hybrid combination of a classical computer and a quantum computer].”)

Wiebe teaches wherein the control system is configured to receive input data at the classical computer from the real physical system, wherein the classical computer and the quantum computer are configured to exchange data therebetween,
(Wiebe, “[0075] FIG. 11 and the following discussion are intended to provide a brief, general description of an exemplary computing environment in which the disclosed technology may be implemented... Typically, a classical computing environment [wherein the control system is configured to receive input data at the classical computer from the real physical system ie data from the one or more monitoring/measurement devices 1046 of fig. 10] is coupled to a quantum computing environment [wherein the classical computer and the quantum computer are configured to exchange data therebetween ie coupled], but a quantum computing environment is not shown in FIG. 11.”)

Wiebe teaches and to use a variational inference arrangement executed on the hybrid combination to process the input data to generate corresponding output data from the classical computer for use in controlling or monitoring operation of the real physical system,
(Wiebe, “[0073] With reference to FIG. 10, an exemplary system for implementing some aspects of the disclosed technology includes a computing environment 1000 [use a variational inference arrangement] that includes a quantum processing unit 1002 and one or more monitoring/measuring device(s) 1046 [executed on the hybrid combination to process the input data to generate corresponding output data from the classical computer for use in controlling or monitoring operation of the real physical system]. The quantum processor executes quantum circuits (such as the circuit of FIG. 2) that are precompiled by classical compiler unit 1020 utilizing one or more classical processor(s) 1010.”)
Wiebe teaches wherein the variational inference arrangement is implemented at least in part by using at least one Bayesian network model arrangement [using a Born machine implemented] using the quantum computer
(Wiebe, “[0021] The disclosure generally pertains to methods and apparatus for performing Bayesian inference on a quantum computer [wherein the variational inference arrangement is implemented at least in part by using at least one Bayesian network model arrangement ie Bayesian inference … using the quantum computer] using quantum rejection sampling. The disclosed approaches typically permit Bayesian inference to be performed on a quantum computer with high probability of success, unlike conventional approaches.”)

However, Wiebe does not explicitly teach
[using at least one Bayesian network model arrangement] implemented using a Born machine [implemented using the quantum computer.]
Benedetti teaches [using at least one Bayesian network model arrangement] implemented using a Born machine
(Bendetti, Section 2, “Consider the problem of generating a pure state ρg close to an unknown pure target state ρt, where closeness is measured with respect to some distance metric to be chosen. Hereby we use subscripts g and t to label ‘generated’ and ‘target’ states, respectively. The unknown target state is provided a finite number of times by a channel. If we were able to learn the state preparation procedure, then we could generate as many ‘copies’ as we want and use these in a subsequent application. We now describe a game between two players whose outcome is an approximate state preparation for the target state.
Borrowing language from the literature of adversarial machine learning, the two players are called the generator and the discriminator. The task of the generator is to prepare a quantum state and fool the other player into thinking that it is the true target state. Thus, the generator is a unitary transformation G applied to some known initial state, say
PNG
media_image2.png
32
205
media_image2.png
Greyscale
.We will discuss the generator’s strategy later.
The discriminator has the task of distinguishing between the target state and the generated state. It is presented with the mixture
PNG
media_image3.png
26
189
media_image3.png
Greyscale
, where P(t) and P(g) are prior probabilities summing to one. Note that in practice the discriminator sees one input at a time rather than the mixture of density matrices, but we can treat the uncertainty in the input state using this picture. The discriminator performs a positive operator-valued measurement (POVM) {Eb} on the input, so that
PNG
media_image4.png
25
78
media_image4.png
Greyscale
. According to Born’s rule [using a Born machine], measurement outcome b is observed with probability
PNG
media_image5.png
29
140
media_image5.png
Greyscale
. The outcome is then fed to a decision rule, a function that estimates which of the two states was provided in input. A straightforward application of Bayes’ theorem [using at least one Bayesian network model arrangement implemented] suggests that the decision rule should select the label for which the posterior probability is maximal
PNG
media_image6.png
32
184
media_image6.png
Greyscale
This rule is called the Bayes’ decision function and is optimal in the sense that, given an optimal POVM, any other decision function has a larger probability of error [19].”)
Wiebe and Benedetti are both considered to be analogous to the claimed invention because they are in the same field of performing inferencing on quantum computers. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wiebe to incorporate the teachings of Benedetti in order to provide a successful method of modeling high-dimensional probability distributions for use in quantum computing and a resilient backpropagation algorithm to perform efficient optimization (Benedetti, Abstract, “Adversarial learning is one of the most successful approaches to modeling high-dimensional probability distributions from data. The quantum computing community has recently begun to generalize this idea and to look for potential applications. In this work, we derive an adversarial algorithm for the problem of approximating an unknown quantum pure state. Although this could be done on universal quantum computers, the adversarial formulation enables us to execute the algorithm on near-term quantum computers. Two parametrized circuits are optimized in tandem: one tries to approximate the target state, the other tries to distinguish between target and approximated state. Supported by numerical simulations, we show that resilient backpropagation algorithms perform remarkably well in optimizing the two circuits. We use the bipartite entanglement entropy to design an efficient heuristic for the stopping criterion. Our approach may find application in quantum state tomography.”)

In regards to claim 16,
Wiebe and Benedetti teaches The control system of claim 15,
Bendetti teaches wherein the Born machine is configured to generate one or more potential Bayesian network models [representative of the real physical system] based on prior data and posterior data [obtained from the real physical system],
(Bendetti, Section 2, “Consider the problem of generating a pure state ρg close to an unknown pure target state ρt, where closeness is measured with respect to some distance metric to be chosen. Hereby we use subscripts g and t to label ‘generated’ and ‘target’ states, respectively. The unknown target state is provided a finite number of times by a channel. If we were able to learn the state preparation procedure, then we could generate as many ‘copies’ as we want and use these in a subsequent application. We now describe a game between two players whose outcome is an approximate state preparation for the target state.
Borrowing language from the literature of adversarial machine learning, the two players are called the generator and the discriminator. The task of the generator is to prepare a quantum state and fool the other player into thinking that it is the true target state. Thus, the generator is a unitary transformation G applied to some known initial state, say
PNG
media_image2.png
32
205
media_image2.png
Greyscale
.We will discuss the generator’s strategy later.
The discriminator has the task of distinguishing between the target state and the generated state. It is presented with the mixture
PNG
media_image3.png
26
189
media_image3.png
Greyscale
, where P(t) and P(g) are prior probabilities [based on prior data] summing to one. Note that in practice the discriminator sees one input at a time rather than the mixture of density matrices, but we can treat the uncertainty in the input state using this picture. The discriminator performs a positive operator-valued measurement (POVM) {Eb} on the input, so that
PNG
media_image4.png
25
78
media_image4.png
Greyscale
. According to Born’s rule, measurement outcome b is observed with probability
PNG
media_image5.png
29
140
media_image5.png
Greyscale
. The outcome is then fed to a decision rule, a function that estimates which of the two states was provided in input. A straightforward application of Bayes’ theorem suggests that the decision rule should select the label for which the posterior probability [and posterior data] is maximal
PNG
media_image6.png
32
184
media_image6.png
Greyscale
[wherein the Born machine is configured to generate one or more potential Bayesian network models] This rule is called the Bayes’ decision function and is optimal in the sense that, given an optimal POVM, any other decision function has a larger probability of error [19].”)
Benedetti teaches and the at least one Bayesian network model arrangement is configured to converge from the one or more potential Bayesian network models to an optimal Bayesian network model [to use to control or monitor the real physical system.]
(Benedetti, Section 2.2, “A family of optimizers known as resilient backpropagation algorithms (Rprop) [17] is particularly well suited for problems where the error surface is characterized by large plateaus with small gradient. Rprop algorithms adapt the step size for each parameter based on the agreement between the sign of its current and previous partial derivatives. If the signs of the two derivatives agree, then the step size for that parameter is increased multiplicatively. This allows the optimizer to traverse large areas of small gradient with an increasingly high speed. If the signs disagree, it means that the last update for that parameter was large enough to jump over a local minima. To fix this, the parameter is reverted to its previous value and the step size is decreased multiplicatively. Rprop is therefore resilient to gradients with very small magnitude as long as the sign of the partial derivatives can be determined.
We use a variant known as iRprop [the at least one Bayesian network model arrangement is configured to converge (see line 11 of algorithm 1) from the one or more potential Bayesian network models to an optimal Bayesian network model ]− [28] which does not revert a parameter to its previous values when the signs of the partial derivatives disagree. Instead, it sets the current partial derivative to zero so that the parameter is not updated, but its step size is still reduced. The hyperparameters and pseudocode for iRprop− are described in algorithm 1.

PNG
media_image7.png
343
830
media_image7.png
Greyscale
”)
However, Benedetti does not explicitly teach
[wherein the Born machine is configured to generate one or more potential Bayesian network models] representative of the real physical system [based on prior data and posterior data] obtained from the real physical system, [and the at least one Bayesian network model arrangement is configured to converge from the one or more potential Bayesian network models to an optimal Bayesian network model] to use to control or monitor the real physical system.
Wiebe teaches [wherein the Born machine is configured to generate one or more potential Bayesian network models] representative of the real physical system [based on prior data and posterior data] obtained from the real physical system, [and the at least one Bayesian network model arrangement is configured to converge from the one or more potential Bayesian network models to an optimal Bayesian network model] to use to control or monitor the real physical system.
(Wiebe, “[0003] Disclosed herein are methods and apparatus that use quantum computations that permit substantial reductions in computational resources needed to perform Bayesian inference. In typical examples, a set of experimental or other data is obtained, and a prior distribution (or current posterior distribution) associated with the experiment or measurement is obtained [based on prior data and posterior data obtained from the real physical system]. The prior distribution is represented in a quantum register, and a rotation gate is applied to a selected qubit of the quantum register. The state of the selected qubit is measured, and depending on the measurement, the quantum register is associated with an updated posterior, or the prior distribution is represented in the quantum register again. The rotation and measurement operations are repeated until an updated posterior is obtained. This update process can be repeated to produce a final posterior. In some cases, qubit representations of sinc functions are used to estimate distributions, and convolutions of a distribution with a model distribution are determined using quantum computations to permit Bayesian inference [to use to control or monitor the real physical system] for time-varying systems.”)

In regards to claim 17,
Wiebe and Benedetti teach The control system of claim 16,
Benedetti teaches wherein the at least one Bayesian network model arrangement is configured to converge from the one or more potential Bayesian network models in a repeated manner to the optimal Bayesian network model to use.
(Benedetti, Section 2.2, “A family of optimizers known as resilient backpropagation algorithms (Rprop) [17] is particularly well suited for problems where the error surface is characterized by large plateaus with small gradient. Rprop algorithms adapt the step size for each parameter based on the agreement between the sign of its current and previous partial derivatives. If the signs of the two derivatives agree, then the step size for that parameter is increased multiplicatively. This allows the optimizer to traverse large areas of small gradient with an increasingly high speed. If the signs disagree, it means that the last update for that parameter was large enough to jump over a local minima. To fix this, the parameter is reverted to its previous value and the step size is decreased multiplicatively. Rprop is therefore resilient to gradients with very small magnitude as long as the sign of the partial derivatives can be determined.
We use a variant known as iRprop [wherein the at least one Bayesian network model arrangement is configured to converge from the one or more potential Bayesian network models in a repeated manner (see lines 1-2 in algorithm 1 wherein in a repeated manner is iterating and line 1 repeat is specifically directed to repetition until convergence) to the optimal Bayesian network model to use ie until convergence]− [28] which does not revert a parameter to its previous values when the signs of the partial derivatives disagree. Instead, it sets the current partial derivative to zero so that the parameter is not updated, but its step size is still reduced. The hyperparameters and pseudocode for iRprop− are described in algorithm 1.

PNG
media_image7.png
343
830
media_image7.png
Greyscale
”)
Claims 23, 31-32, and 34 are rejected on the same grounds under 35 U.S.C. 103 as claim 15
Claim 24 is rejected on the same grounds under 35 U.S.C. 103 as claim 16
Claim 25 is rejected on the same grounds under 35 U.S.C. 103 as claim 17

Claim(s) 18-20 and 26-28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiebe in view of Benedetti in further view of Ghahramani, Zoubin, and Michael Jordan. "Factorial hidden Markov models” (“Ghahramani”)
In regards to claim 18,
Wiebe and Benedetti teach The control system of claim 15,
Ghahramani teaches wherein the at least one Bayesian network model arrangement comprises a nested series of models, wherein at least one of the models of the nested series models,
(Ghahramani, Section 3, “Unfortunately the exact E step for factorial HMMs is computationally intractable. This fact can best be shown by making reference to standard algorithms for probabilistic inference in graphical models (Lauritzen & Spiegelhalter, 1988), although it can also be derived readily from direct application of Bayes rule [wherein the at least one Bayesian network model arrangement comprises a nested series of models ie factorial HMMs; see fig. 1(b)].”)

PNG
media_image8.png
351
737
media_image8.png
Greyscale

However, Ghahramani does not explicitly teach implemented using the quantum computer

Wiebe teaches is implemented using the quantum computer.
(Wiebe, “[0040] There are many methods for preparing states that represent probability distributions using a quantum computer. By the linearity of quantum mechanics, any method of doing so which is coherent can then be used to prepare a mixture distribution such as is used in SMC approximations. As a result, each can be used in a quantum resampling procedure. Perhaps the most straightforward is the method of Grover and Rudolph which provides an efficient algorithm for preparing a probability distribution that is efficiently integrable. Kitaev and Webb have shown that Gaussian states can be efficiently prepared using a quantum computer [implemented using the quantum computer].”)

Ghahramani is both considered to be analogous to the claimed invention because they are in the same field of variational frameworks comprising of Bayesian networks and hidden Markov models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wiebe and Benedetti to incorporate the teachings of Ghahramani in order to fulfill the need for distributed state representations. (Ghahramani, Section 1, “The need for distributed state representations in HMMs can be motivated in two ways. First such representations let the model automatically decompose the state space into features that decouple the dynamics of the process that generated the data. Second distributed state representations simplify the task of modeling time series that are known a priori to be generated from an interaction of multiple loosely-coupled processes. For example, a speech signal generated by the superposition of multiple simultaneous speakers can be potentially modeled with such an architecture.”)
Examiner’s note: While Ghahramani does not explicitly recite “quantum computing”, Ghahramani discloses “superposition of multiple simultaneous speakers can be potentially modeled with such an architecture” wherein one of ordinary skill in the art could utilize this architecture for quantum computing.

In regards to claim 19,
Wiebe and Benedetti and Ghahramani teaches The control system of claim 18,
Ghahramani teaches wherein the models of the nested series of models are mutually different and are specialized to perform corresponding specialized variational inference functions.
(Ghahramani, Section 3.5, “The approximation presented in the previous section factors the posterior probability such that all the state variables are statistically independent. In contrast to this rather extreme factorization there exists a third approximation that is both tractable and preserves much of the probabilistic structure of the original system. In this scheme the factorial HMM is approximated by M uncoupled HMMs as shown in Figure 2(b). Within each HMM efficient and exact inference is implemented via the forward-backward algorithm [wherein the models of the nested series of models are mutually different ie uncoupled and are specialized to perform corresponding specialized variational inference functions]. The approach of exploiting such tractable substructures was first suggested in the machine learning literature by Saul and Jordan (1996).”)

PNG
media_image9.png
358
741
media_image9.png
Greyscale

In regards to claim 20,
Wiebe and Benedetti and Ghahramani teaches The control system of claim 18,
Ghahramani teaches wherein the nested series of models comprises a nested series of hidden Markov models.
(Ghahramani, Section 3, “Unfortunately the exact E step for factorial HMMs is computationally intractable. This fact can best be shown by making reference to standard algorithms for probabilistic inference in graphical models (Lauritzen & Spiegelhalter, 1988), although it can also be derived readily from direct application of Bayes rule [wherein the nested series of models comprises a nested series of hidden Markov models ie factorial HMMs (hidden Markov models); see fig. 1(b)].”)

Claim 26 is rejected on the same grounds under 35 U.S.C. 103 as claim 18
Claim 27 is rejected on the same grounds under 35 U.S.C. 103 as claim 19
Claim 28 is rejected on the same grounds under 35 U.S.C. 103 as claim 20

Claim(s) 21, 29, 33, and 35 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiebe in view of Benedetti in further view of Zhu, Daiwei, et al. "Training of quantum circuits on a hybrid quantum computer." Science advances 5.10 (2019) (“Zhu”)
In regards to claim 21,
Wiebe and Benedetti teach The control system of claim 16,
Zhu teaches wherein at least one Bayesian network model arrangement of the variational inference arrangement is configured to be taught by using an objective function for at least one of:
(i) minimizing a Kullback-Leibler (KL) divergence of a true posterior and relying on a classifier that estimated a probability ratio; and (ii) teaching using a kernelized Stein discrepancy (KSD) requiring explicit priors and likelihoods, to converge to the optimal Bayesian network model.
(Zhu, pg. 5 col. 2 – pg. 6 col. 1, “BO is a powerful global optimization paradigm. It is best suited to finding optima of multimodal objective functions that are expensive to evaluate. There are two main features that characterize the BO process: the surrogate model and an acquisition function [wherein at least one Bayesian network model arrangement of the variational inference arrangement is configured to be taught by using an objective function for at least one of; note that BO stands for Bayesian Optimization]…
The cost functions used to implement the training are variants of the
original DKL (26)

PNG
media_image10.png
52
377
media_image10.png
Greyscale

Here, p and q are two distributions. DKL(p, q) is an information theoretic measure of how two probability distribution differ [minimizing a Kullback-Leibler (KL) divergence of a true posterior ie q]. If base 2 for the logarithm is used, then it quantifies the expected number of extra bits required to store samples from p when an optimal code designed for q is used instead. It can be shown that DKL(p, q) is nonnegative and is zero if and only if p = q. However, it is asymmetric in the arguments and does not satisfy the triangle inequality. Therefore, DKL(p, q) is not a metric.
Here, we set p as the target distribution [relying on a classifier that estimated a probability ratio; wherein examiner interprets the probability ratio as p (wherein a probability is a ratio) as the KL divergence “relies on” the probability ratio and a classifier can be a person/model that labeled the data to generate said target distribution p (bars-and-stripes states in fig. 1)]. Thus, Eq. 4 is equivalent to
Eq. 3 up to a constant offset, so the optimization of these two functions
is equivalent. Eis a small number (0.0001 here) used to avoid a numerical
singularity when q(i) is measured to be zero. For BO, we used the
clipped symmetrized DKL as the cost function

PNG
media_image11.png
57
425
media_image11.png
Greyscale

This is found to be the most reliable variant of DKL for BO.”)

PNG
media_image12.png
405
416
media_image12.png
Greyscale

Wiebe and Benedetti and Zhu are both considered to be analogous to the claimed invention because they are in the same field of performing inferencing on quantum computers. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wiebe and Benedetti to incorporate the teachings of Zhu in order to provide Bayesian optimization to successfully converge the quantum circuit in a hybrid learning scheme (Zhu, Abstract, “Generative modeling is a flavor of machine learning with applications ranging from computer vision to chemical design. It is expected to be one of the techniques most suited to take advantage of the additional resources provided by near-term quantum computers. Here, we implement a data-driven quantum circuit training algorithm on the canonical Bars-and-Stripes dataset using a quantum-classical hybrid machine. The training proceeds by running parameterized circuits on a trapped ion quantum computer and feeding the results to a classical optimizer. We apply two separate strategies, Particle Swarm and Bayesian optimization to this task. We show that the convergence of the quantum circuit to the target distribution depends critically on both the quantum hardware and classical optimization strategy. Our study represents the first successful training of a high-dimensional universal quantum circuit and highlights the promise and challenges associated with hybrid learning schemes.”)

Claims 29, 33 and 35 are rejected on the same grounds under 35 U.S.C. 103 as claim 21

Claim(s) 22 and 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiebe in view of Benedetti in further view of S. Y. -C. Chen, C. -H. H. Yang, J. Qi, P. -Y. Chen, X. Ma and H. -S. Goan, "Variational Quantum Circuits for Deep Reinforcement Learning," (“Chen”)
In regards to claim 22,
Wiebe and Benedetti teaches The control system of claim 15,
Chen teaches wherein the control system is configured to infer an operating condition of the real physical system from an error signal used to compensate for deviations in operation of the real physical system relative to a learnt representation of the real physical system,
(Chen, Section II, “Reinforcement learning is a machine learning paradigm in which an agent interacts with an environment E over a number of discrete time steps [26]. At each time step t, the agent receives a state or observation st and then chooses an action at from a set of possible actions A according to its policy
π

. The policy is a function mapping the state st to action at. In general, the policy can be stochastic, which means that given a state s, the action output can be a probability distribution [relative to a learnt representation ie probability distribution of the real physical system]. After executing the action at , the agent receives the state of the next time step stC1 and a scalar reward rt . The process continues until the agent reaches the terminal state. An episode is defined as an agent starting from a randomly selected initial state and following the aforementioned process all the way through the terminal state.
Define
PNG
media_image13.png
31
154
media_image13.png
Greyscale
as the total discounted return from time step t, where is the discount factor that lies in (0; 1]. In principle, y is provided by the investigator to control how future rewards are given to the decision-making function. When a large y is considered, the agent takes into account future rewards no matter what a discount rate is. As to a small y, an agent can quickly ignore future rewards within a few time steps. The goal of the agent is to maximize the expected return from each state st in the training process. The action-value function or Q-value function
PNG
media_image14.png
28
99
media_image14.png
Greyscale

PNG
media_image15.png
30
120
media_image15.png
Greyscale
is the expected return for selecting an action a in state s based on policy
π

. The optimal action value function
PNG
media_image16.png
28
204
media_image16.png
Greyscale
gives a maximal action value across all possible policies. The value of state s under policy
π
,

PNG
media_image17.png
26
193
media_image17.png
Greyscale
is the agent's expected return by following policy
π

from the state s. The classical temporal difference (TD) error [26] is used to update value function in reinforcement learning tasks. [wherein the control system is configured to infer an operating condition of the real physical system from an error signal used to compensate for deviations in operation of the real physical system]”)
However, Chen does not explicitly teach wherein the learnt representation of the real physical system is implemented using the at least one Bayesian network model arrangement that is at least partially implemented using the quantum computer

Wiebe teaches wherein the learnt representation of the real physical system is implemented using the at least one Bayesian network model arrangement that is at least partially implemented using the quantum computer.
(Wiebe, “[0040] There are many methods for preparing states that represent probability distributions using a quantum computer [wherein the learnt representation of the real physical system ie probability distribution as given by Chen is implemented using the at least one Bayesian network model arrangement that is at least partially implemented using the quantum computer]. By the linearity of quantum mechanics, any method of doing so which is coherent can then be used to prepare a mixture distribution such as is used in SMC approximations. As a result, each can be used in a quantum resampling procedure.”)
Wiebe and Benedetti and Chen are both considered to be analogous to the claimed invention because they are in the same field of performing inferencing using quantum computers. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wiebe and Benedetti to incorporate the teachings of Chen in order to provide a method of solving the problem of intractability of deep quantum circuits through reinforcement learning (Chen, Abstract, “The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate scale quantum (NISQ) devices. This work explores variational quantum circuits for deep reinforcement learning. Specifically, we reshape classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits. Moreover, we use a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks. To the best of our knowledge, this work is the first proof-of-principle demonstration of variational quantum circuits to approximate the deep Q-value function for decision-making and policy-selection reinforcement learning with experience replay and target network. Besides, our variational quantum circuits can be deployed in many near-term NISQ machines.”)

Claim 30 is rejected on the same grounds under 35 U.S.C. 103 as claim 22

Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pub. No. US20040024750A1: Ulyanov et al. teaches Intelligent mechatronic control suspension system based on quantum soft computing
U.S. Pub. No. US20180247200A1: Rolfe teaches Discrete variational auto-encoder systems and methods for machine learning using adiabatic quantum computers
U.S. Pub. No. US20200134502A1: Anschuetz et al. teaches Hybrid Quantum-Classical Computer System for Implementing and Optimizing Quantum Boltzmann Machines
U.S. Pub. No. US20180165601A1: Wiebe et al. teaches Tomography and generative data modeling via quantum boltzmann training
U.S. Pub. No. US20190378025A1: Gonzalez et al. teaches Quantum feature kernel estimation

Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129

/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129