Patent Application 17072709 - AUTOMATED SETUP AND COMMUNICATION COORDINATION - Rejection
Patent Application 17072709 - AUTOMATED SETUP AND COMMUNICATION COORDINATION
Title: AUTOMATED SETUP AND COMMUNICATION COORDINATION FOR TRAINING AND UTILIZING MASSIVELY PARALLEL NEURAL NETWORKS
Application Information
- Invention Title: AUTOMATED SETUP AND COMMUNICATION COORDINATION FOR TRAINING AND UTILIZING MASSIVELY PARALLEL NEURAL NETWORKS
- Application Number: 17072709
- Submission Date: 2025-05-14T00:00:00.000Z
- Effective Filing Date: 2020-10-16T00:00:00.000Z
- Filing Date: 2020-10-16T00:00:00.000Z
- National Class: 706
- National Sub-Class: 025000
- Examiner Employee Number: 95297
- Art Unit: 2129
- Tech Center: 2100
Rejection Summary
- 102 Rejections: 0
- 103 Rejections: 2
Cited Patents
No patents were cited in this rejection.
Office Action Text
DETAILED ACTION
Examiner Notes
In light of Applicant’s Remarks and Amendments submitted on 01/28/2025, Examiner has withdrawn the objections to the Drawings and Claims and the 112(b) rejections.
Response to Arguments
103
Applicant argues that the prior art of Wu does not teach the amended claim limitations of providing the map of the directed acyclic graph to each vertex of the plurality of vertices of the directed acyclic graph such that each vertex of the plurality of vertices is informed of (i) a flow of data through the plurality of vertices represented by the deterministic topological sort and (ii) an order of traversal of the plurality of vertices of the directed acyclic graph. See pg., 12 of Applicant’s Remarks submitted on 01/28/2025.
To support Applicant’s argument that the prior art of Wu does not teach the above claim limitations, Applicant argues that because “[t]here is no indication in Wu that Wu’s mapping for each vertex is provided to each vertex of Wu’s graph G” the prior art of Wu cannot teach the claim limitation of providing the map of the directed acyclic graph to each vertex of the plurality of vertices of the directed acyclic graph such that each vertex of the plurality of vertices is informed of (i) a flow of data through the plurality of vertices represented by the deterministic topological sort and (ii) an order of traversal of the plurality of vertices of the directed acyclic graph. Id.
Respectfully Examiner disagrees. As the prior art of Wu teaches on page 5 (of the machine translated version) when it comes to constructing the map of the directed acyclic graph comprising the deterministic topological sort, Wu states that “[f]or input data constructed of a directed graph G = (V, E, λ, τ), wherein V is the set of [vertices in] G…given graph G… topological sorting is a mapping point to an integer
γ
:
V
→
[
0
,
1
,
…
,
N
-
1
]
satisfy[ing]
γ
v
=
0
,
∀
v
∈
V
∧
λ
v
._2=true…[i]f the variable total is 0, then…perform[] the operation [first]….”(Emphasis added). Accordingly, the mapping function
γ
takes in an input of vertices V and outputs an execution order/mapping ranging from 0 to N-1 for each vertex in graph G. As detailed by the constraint associated with the mapping function being equal to 0 (i.e.,
γ
v
=
0
) the constraint will be satisfied so long as it applies to all vertices in the set of vertices in the graph G (i.e.,
∀
v
∈
V
) and the function
λ
v
._2 is true. Under the broadest reasonable interpretation (BRI) in light of the Specification, the prior art of Wu teaches that its mapping for each vertex is provided to each vertex of the graph G and thus Wu teaches the above claim limitation.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/17/2024 is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1-2, 4-8, 10-13, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Abadi et al., Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467.2016 Mar 14 (“Abadi”) in view of Wu et al., CN 110941494A(“Wu”) and in view of Suresh et al., Data flow and distributed deep neural network based low latency IoT-edge computation model for big data environment. Engineering Applications of Artificial Intelligence. 2020 Sep 1(“Suresh”).
Regarding claim 1, Abadi teaches a method of distributed computing, the method comprising: dividing a directed acyclic graph among a plurality of nodes, the directed acyclic graph comprising a plurality of vertices linked in pairwise relationships via a plurality of edges, wherein each of the plurality of nodes comprises a computing device(Abadi, pg. 5, see also Fig. 4, “Once the node placement has been computed, the graph is partitioned into a set of subgraphs, one per device[dividing a directed acyclic graph among a plurality of nodes, the directed acyclic graph comprising a plurality of vertices linked in pairwise relationships via a plurality of edges, wherein each of the plurality of nodes comprises a computing device]. Any cross-device edge from x to y is removed and replaced by an edge from x to a new Send node in x’s subgraph and an edge from a corresponding Receive node to y in y’s subgraph. See Figure 4 for an example of this graph transformation.”).
While Abadi teaches directed acyclic graph, Abadi does not teach: performing a topological sort of the vertices of [directed acyclic graph], the topological sort comprising a deterministic topological sort controlling an order of operations for traversing [directed acyclic graph]; generating a map of [directed acyclic graph] that comprises the deterministic topological sort and that comprises an indication of the order of operations for traversing [directed acyclic graph] among the plurality of nodes; providing the map of [directed acyclic graph] to each vertex of the plurality of vertices of [directed acyclic graph] such that each vertex of the plurality of vertices is informed of (i) a flow of data through the plurality of vertices represented by the deterministic topological sort and (ii) an order of traversal of the plurality of vertices [of the directed acyclic graph].1
However, Wu teaches:
performing a topological sort of the vertices of [directed acyclic graph], the topological sort comprising a deterministic topological sort controlling an order of operations for traversing [directed acyclic graph](Wu, pg., 5, “[C]alculating the given graph
G
=
(
V
,
E
,
λ
,
τ
)
, N is the number of vertices in the graph, topological sorting is a mapping point to an integer
γ
:
V
→
[
0
,
1
,
…
,
N
-
1
]
satisfies
γ
v
=
0
,
∀
v
∈
V
⋀
λ
v
…
γ
u
<
γ
v
,
∀
u
,
v
∈
E
∧
λ
v
[performing a topological sort of the vertices, the topological sort comprising a deterministic topological sort]…topological sorting represents execution order operation[controlling an order of operations for traversing]…giving two operation u and v, if
γ
u
<
γ
(
v
)
then executing…u prior to v. [I]f
γ
u
=
γ
v
,
then u and v are preforemd in parallel.”);2
generating a map of [directed acyclic graph] that comprises the deterministic topological sort and that comprises an indication of the order of operations for traversing [directed acyclic graph] among the plurality of nodes(Wu, pg., 5, “[C]alculating the given graph
G
=
(
V
,
E
,
λ
,
τ
)
, N is the number of vertices in the graph, topological sorting is a mapping point to an integer
γ
:
V
→
[
0
,
1
,
…
,
N
-
1
]
[generating a map that comprises the deterministic topological sort and that comprises an indication of the order of operations for traversing] satisfies
γ
v
=
0
,
∀
v
∈
V
⋀
λ
v
…
γ
u
<
γ
v
,
∀
u
,
v
∈
E
∧
λ
v
…topological sorting represents execution order operation…giving two operation u and v, if
γ
u
<
γ
(
v
)
then executing…u prior to v. [I]f
γ
u
=
γ
v
,
then u and v are preforemd in parallel[among the plurality of nodes].”);3
providing the map of [directed acyclic graph] to each vertex of the plurality of vertices of [directed acyclic graph] such that each vertex of the plurality of vertices is informed of (i) a flow of data through the plurality of vertices represented by the deterministic topological sort and (ii) an order of traversal of the plurality of vertices [of the directed acyclic graph] (Wu, pg., 5, “For input data constructed of a directed graph G = (V, E, λ, τ), wherein V is the set of G in the edgeset, E = V
×
V is G, λ: V
→
(O, Bool) represents the each vertex mapped to a Boolean function of the operation
o
∈
O
and operation is parameterized… topological sorting is a mapping point to an integer
γ
:
V
→
[
0
,
1
,
…
,
N
-
1
]
satisfy[ing]
γ
v
=
0
,
∀
v
∈
V
∧
λ
v
._2=true…[i]f the variable total is 0, then…perform[] the operation [first][providing the map to each vertex of the plurality of vertices such that each vertex of the plurality of vertices is informed of (i) a flow of data through the plurality of vertices represented by the deterministic topological sort and (ii) an order of traversal of the plurality of vertices]….”);4
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Abadi with the teachings of Wu the motivation to do so would be to increase the training speed of deep learning models that are executed on GPUs(Wu, pg., 2, “The present invention thus provides a computation graph modeling techniques…research of GPU memory and external memory exchanging data… provides a data processing method capable of fusion, to input data of different data structure, the operation then performs model capable of performing data pre-processing to obtain the data of the same size according to the model, so it can increase the training speed of the processing speed”).
While Abadi in view of Wu teaches [the directed acyclic graph], Abadi in view of Wu does not teach: training a neural network associated with a vehicle system by using a distributed computing system to traverse [the directed acyclic graph]; and executing the trained neural network to facilitate a human machine interface of the vehicle system by generating one or more predictions associated with the vehicle system to simulate a behavior or an attribute of the vehicle system. 5
However, Suresh teaches:
training a neural network associated with a vehicle system by using a distributed computing system to traverse [the directed acyclic graph](Suresh, pg., 3, see also figs. 2, 3, and 4, “[T]he information is gathered using the AI filter which avoids the unwanted data in the gathering stages itself. This enables us to achieve high accuracy using the Deep Neural Network. In the proposed, system takes the 12 inputs viz., the event time, truck id, driver id, route id, latitude, longitude, speed, foggy, rainy, and windy to the Deep Neural Network[training a neural network associated with a vehicle system]… [a] Storm application is outlined as a ‘‘topology’’ in the state of a coordinated non-cyclic chart (DAG) with spouts and jolts going about as the diagram vertices. Edges on the map are named streams and direct data starting with one hub then onto the next. Together, the topology goes about as a data change pipeline. Storm topologies run inconclusively until it is executed, while a MapReduce work DAG should run at the end[by using a distributed computing system to traverse].”);6
and executing the trained neural network to facilitate a human machine interface of the vehicle system by generating one or more predictions associated with the vehicle system to simulate a behavior or an attribute of the vehicle system(Suresh, pg., 6, see also Table 2, “This proposed system is implemented using three different components; the first component is IoT truck simulator by Horton works which generates the data, the second component is Horton Data Flow (HDF) which contains the various data flow tools and the final component is Python code which is used to link the above parts and implement the distributed deep neural network[and executing the trained neural network]… [t]he data collected by the IoT devices are sent to the flow management tools for real-time stream processing and analytics through machine learning. This will help in predictions such as traffic congestion on different routes, truck monitoring, and driver behavior monitoring, which are done in the cloud using HDF[to facilitate a human machine interface of the vehicle system by generating one or more predictions associated with the vehicle system to simulate a behavior or an attribute of the vehicle system].” ).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Abadi in view of Wu with the teachings of Suresh the motivation to do so would be to combined the predictions generated by distributed deep learning done at the cloud with data collected from IoT-Edge environments(Suresh, pgs. 1-2, “IoT represents a general idea for the flexibility of network devices to sense and collect information from around the globe and share that information across a network where it will be processed and used for numerous exciting commitments… [i]n IoT, data needs to be analyzed rapidly… [t]his travel time is crucial for applications like monitoring of in-flight jet engines or the operation of driverless cars where even millisecond delay is unacceptable… [t]he specific research objective of this paper is to develop an approach for the integration of data flow and distributed deep learning in the IoT-Edge environment to bring down the latency and increase accuracy starting from the data generation phase.”).
Regarding claim 2, Abadi in view of Wu and in view of Suresh teach the method of claim 1, wherein the directed acyclic graph forms a part of the neural network(Abadi, pgs., 2-3, see also figs. 1 and 2, “A TensorFlow computation is described by a directed graph, which is composed of a set of nodes. The graph represents a dataflow computation… [a]n example fragment to construct and then execute a TensorFlow graph using the Python front end is shown in Figure 1, and the resulting computation graph in Figure 2[wherein the directed acyclic graph forms a part of the neural network].” & Abadi, pg., 3, As figure 2 details below:
PNG
media_image1.png
436
498
media_image1.png
Greyscale
The DAG represents a one-layer neural network in which the input x is multiplied by the weight matrix W to ultimately compute the following output relu = tf.nn.relu(tf.matmul(W, x) + b) (as detailed by the code in fig. 1)).
Regarding claim 4, Abadi in view of Wu and in view of Suresh teach the method of claim 2, further comprising: identifying an edge linking separate nodes; and inserting a data exchange vertex between the separate nodes(Abadi, pg., 5, see also fig. 4, “Once the node placement has been computed, the graph is partitioned into a set of subgraphs, one per device. Any cross-device edge from x to y [identifying an edge linking separate nodes] is removed and replaced by an edge from x to a new Send node in x’s subgraph and an edge from a corresponding Receive node to y in y’s subgraph[and inserting a data exchange vertex between the separate nodes].”).
Regarding claim 5, Abadi in view of Wu and in view of Suresh teach the method of claim 4, wherein identifying the edge linking separate nodes comprises:
tracing an edge from a first vertex to a second vertex, wherein the edge coupled the first vertex to the second vertex; comparing a node of the first vertex to a node of the second vertex; and determining that the edge links separate nodes when the node of the first vertex is different than the node of the second vertex(Abadi, pg., 5, As Figure 4 details below:
PNG
media_image2.png
286
506
media_image2.png
Greyscale
In this case before insertion of the send/receive nodes i.e., the left sided portion of fig. 4, the vertex x is located on Device A and the vertex y is located on Device B[tracing an edge from a first vertex to a second vertex, wherein the edge coupled the first vertex to the second vertex; comparing a node of the first vertex to a node of the second vertex;]. The directed edge connecting vertex x to vertex y lies on different hardware devices[and determining that the edge links separate nodes when the node of the first vertex is different than the node of the second vertex].).
Regarding claim 6, Abadi in view of Wu and in view of Suresh teach the method of claim 5, wherein the data exchange vertex comprises a recursive directed acyclic graph structure comprising: a send vertex located on the node of the first vertex; and a receive vertex located on the node of the second vertex, wherein when the data exchange vertex is activated, both the send vertex and the receive vertex run in concert(Abadi, pg., 5, see also fig. 4, “Any cross-device edge from x to y is removed and replaced by an edge from x to a new Send node[a send vertex located on the node of the first vertex] in x’s subgraph [a recursive directed acyclic graph structure] and an edge from a corresponding Receive node[and a receive vertex located on the node of the second vertex] to y in y’s subgraph [a recursive directed acyclic graph structure]. See Figure 4 for an example of this graph transformation. At runtime, the implementations of the Send and Receive nodes coordinate to transfer data across devices[wherein when the data exchange vertex is activated, both the send vertex and the receive vertex run in concert].”).
Regarding claim 7, Abadi in view of Wu and in view of Suresh teach the method of claim 6, further comprising: activating a data vertex; and running the send vertex and the receive vertex in concert while passing a tensor from the send vertex to the receive vertex(Abadi, pg., 5, see also fig. 4, “At runtime, the implementations of the Send and Receive nodes coordinate to transfer data across devices[and running the send vertex and the receive vertex in concert while passing a tensor from the send vertex to the receive vertex]… [w]hen we insert Send and Receive nodes, we canonicalize all users of a particular tensor on a particular device to use a single Receive node[activating a data vertex], rather than one Receive node per downstream user on a particular device. This ensures that the data for the needed tensor is only transmitted once between a source device
→
destination device pair.”).
Regarding claim 8, Abadi in view of Wu and in view of Suresh teach the method of claim 7, wherein passing the tensor from the send vertex to the receive vertex comprises: passing a series of communications characterizing the tensor from the send vertex to the receive vertex; and subsequently passing the tensor from the send vertex to the receive vertex(Abadi, pg., 5, see also fig. 4, “At runtime, the implementations of the Send and Receive nodes coordinate to transfer data across devices… [w]hen we insert Send and Receive nodes, we canonicalize all users of a particular tensor on a particular device to use a single Receive node[passing a series of communications characterizing the tensor from the send vertex to the receive vertex], rather than one Receive node per downstream user on a particular device. This ensures that the data for the needed tensor is only transmitted once between a source device
→
destination device pair[and subsequently passing the tensor from the send vertex to the receive vertex].”).
Regarding claim 10, Abadi in view of Wu and in view of Suresh teach the method of claim 1, further comprising: creating at least one clone directed acyclic graph identical to the directed acyclic graph, the clone directed acyclic graph comprising a plurality of clone vertices; for each of the clone vertices, identifying a corresponding vertex in the directed acyclic graph; during training of the directed acyclic graph and the clone directed acyclic graph, calculating aggregate gradient data based on gradient data from each of the clone vertices and its corresponding vertex in the directed acyclic graph; and updating at least one weight of the directed acyclic graph and the clone directed acyclic graph based on the aggregate gradient data(Abadi, pg., 6, see also fig. 5, “Many optimization algorithms, including common machine learning training algorithms like stochastic gradient descent[during training]… compute the gradient of a cost function with respect to a set of inputs…[w]hen TensorFlow needs to compute the gradient of a tensor C with respect to some tensor I on which C depends, it first finds the path in the computation graph from I to C. Then it backtracks from C to I, and for each operation on the backward path it adds a node to the TensorFlow graph[creating at least one clone directed acyclic graph identical to the directed acyclic graph, the clone directed acyclic graph comprising a plurality of clone vertices; for each of the clone vertices, identifying a corresponding vertex in the directed acyclic graph], composing the partial gradients along the backwards path using the chain rule. The newly added node computes the “gradient function” for the corresponding operation in the forward path… [t]his function takes as input not only the partial gradients computed already along the backward path, but also, optionally, the inputs and outputs of the forward operation[during training of the directed acyclic graph and the clone directed acyclic graph, calculating aggregate gradient data based on gradient data from each of the clone vertices and its corresponding vertex in the directed acyclic graph]” & Abadi, pg., 6, As Figure 5 details below:
PNG
media_image3.png
373
456
media_image3.png
Greyscale
The graph on the left represents the DAG and the graph on the right represents the cloned DAG. The cloned DAG computes the partial gradient of the cost function with respect to the weight matrix i.e.,
d
C
d
W
and is accessed by the DAG through the directed edge connecting the nodes W and x to the node dMatMul [and updating at least one weight of the directed acyclic graph and the clone directed acyclic graph based on the aggregate gradient data]).
Regarding claim 11, Abadi in view of Wu and in view of Suresh teach the method of claim 1,
wherein one of the plurality of vertices of the directed acyclic graph comprises
an entry vertex(Abadi, pg., 7, see also fig. 6, “Each node:port specified in inputs is replaced with a feed node[an entry vertex], which will pick up the provided input tensor from specially-initialized entries in a Rendezvous object used for the Run call.”),
the method further comprising:
identifying the nodes underlying the directed acyclic graph(Abadi, pg., 7, see also fig. 6, “[O]nce the client has set up a computation graph in a Session, our Run method allows them to execute an arbitrary subgraph of the whole graph[identifying the nodes underlying the directed acyclic graph]”);
generating a subordinate directed acyclic graph in the entry vertex, the subordinate directed acyclic graph comprising a plurality of subordinate vertices, each of the plurality of subordinate vertices corresponding to a one of the nodes underlying the directed acyclic graph(Abadi, pg., 7, see also fig. 6, “Each node:port specified in inputs is replaced with a feed node[generating a subordinate directed acyclic graph in the entry vertex], which will pick up the provided input tensor from specially-initialized entries in a Rendezvous object used for the Run call… once the graph has been rewritten with the insertion of these special feed and fetch nodes, the set of nodes to execute can be determined by starting at each of the nodes named by any output and working backwards in the graph using the graph dependencies to determine the full set of nodes that must be executed in the rewritten graph in order to compute the outputs[the subordinate directed acyclic graph comprising a plurality of subordinate vertices, each of the plurality of subordinate vertices corresponding to a one of the nodes underlying the directed acyclic graph].”);
receiving data and metadata at the entry vertex; delivering the data to a next vertex in the directed acyclic graph; and communicating the metadata to nodes underlying the directed acyclic graph via the subordinate directed acyclic graph(Abadi, pg., 7, see also fig. 6, First, the Run call accepts inputs, an optional mapping of name:port names to “fed” tensors values[receiving data and metadata at the entry vertex]. Second, the Run call accepts output names, a list of output name[:port] specifications indicating which nodes should be executed, and, if the port portion is present in a name, that that particular output tensor value for the node should be returned to the client if the Run call completes successfully[delivering the data to a next vertex in the directed acyclic graph]. The graph is transformed based on the values of inputs and outputs. Each node:port specified in inputs is replaced with a feed node[the entry vertex], which will pick up the provided input tensor from specially-initialized entries in a Rendezvous object used for the Run call… [f]igure 6 shows an original graph on the left, and the transformed graph that results when Run is invoked with inputs=={b} and outputs=={f:0}[ and communicating the metadata to nodes underlying the directed acyclic graph via the subordinate directed acyclic graph].”).
Regarding claim 12 Abadi in view of Wu and in view of Suresh teach a system comprising: one or more processors; and one or more memories storing computer-executable instructions that, when executed by the one or more processors, configure the one or more processors to(Suresh, pg., 6, “Table 2 shows the simulation parameters used
in this research work.
PNG
media_image4.png
426
607
media_image4.png
Greyscale
of which the edge gateway configuration consists of 2 Intel E5645 Processor with 24 GB of memory) and for all other claim elements of claim 12 they are rejected on the same basis as claim 1 since they are analogous claims.
Referring to dependent claims 13 and 15-19 they are rejected on the same basis as
dependent claims 2 and 4-8 since they are analogous claims.
Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Abadi et al., Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467. 2016 Mar 14 (“Abadi”) in view of Wu et al., CN 110941494A(“Wu”) and in view of Suresh et al., Data flow and distributed deep neural network based low latency IoT-edge computation model for big data environment. Engineering Applications of Artificial Intelligence. 2020 Sep 1(“Suresh”) and further in view of Mayer et al., "The tensorflow partitioning and scheduling problem: it's the critical path!." Proceedings of the 1st Workshop on Distributed Infrastructures for Deep Learning. 2017(“Mayer”).
Regarding claim 9, Abadi in view of Wu and in view of Suresh teach the method of claim 8, wherein passing the series of communications characterizing the tensor from the send vertex to the receive vertex comprises:
passing at least one second value characterizing data contained in the tensor from the send vertex to the receive vertex(Abadi, pg., 5, see also fig. 4, “When we insert Send and Receive nodes, we canonicalize all users of a particular tensor on a particular device to use a single Receive node, rather than one Receive node per downstream user on a particular device. This ensures that the data for the needed tensor is only transmitted once between a source device
→
destination device pair[passing at least one second value characterizing data contained in the tensor from the send vertex to the receive vertex]….”);
and passing a third value directing tracking of a gradient of the Neural Network(Abadi, pg., 6, see also fig. 2 and 5, “When TensorFlow needs to compute the gradient of a tensor C with respect to some tensor I on which C depends, it first finds the path in the computation graph from I to C. Then it backtracks from C to I, and for each operation on the backward path it adds a node to the TensorFlow graph, composing the partial gradients along the backwards path using the chain rule[and passing a third value directing tracking of a gradient]. The newly added node computes the “gradient function” for the corresponding operation in the forward path… Figure 5 shows gradients for a cost computed from the example of Figure 2[of the Neural Network].”).
Abadi in view of Wu and in view of Suresh do not teach: passing a first value indicating a rank of the tensor from the send vertex to the receive vertex; passing a vector of values indicating a shape of the tensor from the send vertex to the receive vertex.
However, Mayer teaches:
passing a first value indicating a rank of the tensor from the send vertex to the receive vertex (Mayer, pg., 5, see also Figure 1, “[I]t is the edge weight (size of the tensor) [rank of the tensor] divided by the transfer rate between the two devices.
P
C
T
v
i
=
max
v
j
∈
succ
(
v
i
)
(
P
C
T
v
j
+
t
r
a
n
s
v
j
,
v
i
)
+
c
i
S
p
(
v
i
)
[passing a first value indicating a rank of the tensor from the send vertex to the receive vertex]”);
passing a vector of values indicating a shape of the tensor from the send vertex to the receive vertex (Mayer, pg. 5, see also Figure 1 and Table 1, “Tensor sizes were again randomly set between 1 and 100 bytes[passing a vector of values indicating a shape of the tensor from the send vertex to the receive vertex]. The number of operations to compute a vertex function was randomly set between 1 and 100 operations.”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Abadi in view of Wu and in view of Suresh and further in view of Mayer the motivation to do so would be to yield the best results while focusing on the minimization of the execution time of the critical path to outperform other strategies by up to a factor of 4 (Mayer, pg., 6, “The evaluation results based on a simulation indicate that those partitioning and scheduling strategies yield the best results that focus on the minimization of the execution time of the critical path, outperforming other strategies by up to a factor of 4”).
Referring to dependent claim 20 it is rejected on the same basis as dependent claim 9 since they are analogous claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADAM C STANDKE whose telephone number is (571)270-1806. The examiner can normally be reached Gen. M-F 9-9PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADAM C STANDKE/
Assistant Examiner
Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
1 Examiner Notes: Brackets and non-bolded text represent claim elements that have already been taught by the prior art of Abadi.
2 Examiner Notes: Examiner used both the machine translation and the untranslated version for the mathematical formulas
3 Examiner Notes: Examiner used both the machine translation and the untranslated version for the mathematical formulas
4 Examiner Notes: Examiner used both the machine translation and the untranslated version for the mathematical formulas
5 Examiner Notes: Brackets and non-bolded text represent claim elements that have already been taught by the prior art of Abadi.
6 Examiner Notes: Brackets and non-bolded text represent claim elements that have already been taught by the prior art of Abadi.