Jump to content

Patent Application 17509507 - FEDERATED LEARNING DATA SOURCE SELECTION - Rejection

From WikiPatents

Patent Application 17509507 - FEDERATED LEARNING DATA SOURCE SELECTION

Title: FEDERATED LEARNING DATA SOURCE SELECTION

Application Information

  • Invention Title: FEDERATED LEARNING DATA SOURCE SELECTION
  • Application Number: 17509507
  • Submission Date: 2025-05-20T00:00:00.000Z
  • Effective Filing Date: 2021-10-25T00:00:00.000Z
  • Filing Date: 2021-10-25T00:00:00.000Z
  • National Class: 706
  • National Sub-Class: 012000
  • Examiner Employee Number: 99364
  • Art Unit: 2127
  • Tech Center: 2100

Rejection Summary

  • 102 Rejections: 0
  • 103 Rejections: 5

Cited Patents

No patents were cited in this rejection.

Office Action Text


    Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .

Response to Amendment
	The amendment filed 03/18/2025 has not been entered. Claims 1-20 remain pending in the application.

Response to Arguments
	Applicant’s argument filed 03/18/2025 have been fully considered but they are not persuasive.

	Applicant’s Argument: On page 11 of Applicant’s response, applicant states “Applicant submits that the claimed “computing . . . an influential score” feature does not recite “mathematical concepts.” MPEP §2106.04(a)(2)(1) states, “[w]hen determining whether a claim recites a mathematical concept (1.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), examiners should consider whether the claim recites a mathematical concept or merely limitations that are based on or involve a mathematical concept. A claim does not recite a mathematical concept . . . if it is only based on or involves a mathematical concept.” (Emphasis added). 
The July 2024 Subject Matter Eligibility Examples (“2024 SME Examples”) further illustrates the MPEP §2106.04(a)(2)(I) analysis. With reference to Example 48 (claim 3), the 2024 SME Examples finds no recitation of mathematical concepts, stating, “Step (e) requires converting a cluster of points in feature space into speech signals in the time domain... . [W]hile the conversion may be based on mathematical concepts, the claim does not specify any mathematical formulae, calculations, or relationships.” (2024 SME Examples, p. 26; emphasis added). As such, the mere fact that a claim involves “computing . . . an influential score” cannot lead to a determination that the claim recites mathematical concepts.”

	Examiner’s Response: Applicant’s argument is not persuasive. The claim limitation of “computing ... an influential score” is a mathematical calculation itself and not only based on a mathematical concept. In the Specification (par. 45), the determination of the “influence can be computed through computation of the influential score”. The recited claim limitation is directed to an abstract idea because it is a step of performing a mathematical calculation.

Applicant’s Argument: On page 11 and 12 of Applicant’s response, applicant states “Applicant has clarified the previously presented “selecting” feature to recite, “updating ... communication with the central server to a subset of the plurality of data sources,” and has clarified the previously presented “generating” feature to recite, “iteratively generating ... the training dataset utilizing updated data.” (Emphasis added). “[E]xamples of mental processes include observations, evaluations, judgments, and opinions.” (MPEP § 2106.04(a)(2)(I)). Applicant submits that the human mind (even if aided with pen and paper) cannot practically perform the claimed, “updating” and “iteratively generating” features using “observations, evaluations, judgments, and opinions.” As such, amended independent claim 1 does not recite a mental process. The features recited in amended claim | are respectfully believed to bring independent claims 1, 11, and 12, and any dependent claims, within the standard for eligibility under Step 2A-Prong One. Therefore, Applicant respectfully requests reconsideration and withdrawal of the present rejections under 35 U.S.C. § 101.”

	Examiner’s Response: Applicant’s argument is not persuasive. The amended claims do not overcome the determination that the claim recites an abstract idea under subject matter eligibility analysis in step 2A prong 1. The claim limitation “updating ... communication with the central server” still recites an abstract idea of a mental process that can be perform in the human mind. The process of updating communication may be similar to the determination of where to source the supplies from for a supermarket and the determination may be based on the reputation of the third-party supplier. A supermarket may want to continue to conduct business with suppliers with high reputation and the supermarket may also decide to reduce the communication with suppliers with low reputation scores in sourcing supplies from them. 
The claim limitation “iteratively generating ... updated data” still recites an abstract idea of a mental process that can be perform in the human mind. In the given example, a supermarket may continuously update their inventory list of supplies as the supermarket is receiving the goods from external third-party suppliers.
	Therefore, the claimed invention as a whole is directed to an abstract idea and it is subject matter ineligible. 

Applicant’s Argument: On page 13 and 14 of Applicant’s response, applicant states “The Office alleges that Mars discloses the claimed “validation dataset having a plurality of annotated datapoints” by disclosing that “seed samples may include _one or more sample queries or_ sample prompts that may be used as input examples for obtaining machine learning training data from one or more external training data sources.” (Office Action, pp. 9 and 10; emphasis added). Applicant respectfully disagrees. Mars discloses that “a first training data seed sample may be the query of “How much is a medium pizza?’ and a second training data seed sample may be the prompt of ‘Get me a large pizza.’ These training data seed samples may be input into a first user interface of the machine learning configuration and management console.” (Mars, col. 10, lines 14-19; emphasis added). Neither of the above “seed sample” examples disclose the claimed “validation dataset having a plurality of annotated datapoints.” (Emphasis added). Instead, the “seed samples” are merely “sample _queries or sample prompts” that are “input examples for obtaining machine learning training data from one or more external training data sources.” (Mars, col. 10, lines 11-14; emphasis added).”

	Examiner’s Response: Applicant’s argument is not persuasive. The training data seed samples are user queries based on a type of training data that an administrator desires to receive from one or more external data sources (col. 10, lines 6-19). Additionally, the Mars (col. 4, lines 20-39; col. 5, lines 30-53) discloses a classification engine that may generate a classification label for the user query and a slot identification engine that generates slot labels for the user input queries. Therefore, Mars teaches that the seed samples are a plurality of annotated datapoints.

Applicant’s Argument: On page 14 of Applicant’s response, applicant states “Mars also fails to disclose the claimed, “a respective influential score of a data source identifies _an_ influence of the data source in accurately predicting annotations of the validation dataset _when the respective data_is used in the machine-learning model.” (Emphasis added). At best, Mars discloses “level of quality preferably relates to an accuracy of labels generated by the external training data source for each labeled training data sample provided thereby.” (Mars, col. 12, lines 23-25; emphasis added). Since Mars fails to disclose the claimed “validation dataset” as noted above, Mars necessarily fails to disclose “influential score of a data source... in accurately predicting annotations of the validation dataset when the data is used in the machine-learning model,” as recited in independent claim 1. (Emphasis added).”

	Examiner’s Response: Applicant’s argument is not persuasive. Mars discloses (col. 12, lines 29-35) a quality score may be generated for individual external data sources. In the given example, a data source may have a quality level of 8 and another data source may have a low-quality level of 2. The quality score is based on an accuracy of labels generated by the external training data source. As explained above, the seed samples are the user query input data that seeks additional training data for a classification task by the model.

Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.


Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.


Regarding Claim 1:

Subject Matter Eligibility Analysis Step 1:

Claim 1 recites “A method, comprising” and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1: 
“computing, ” (a mathematical calculation)
“updating, ” (a mental process, i.e. judgement)
“iteratively generating, ” (a mental process, i.e. judgement)
Claim 1 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2: 
"receiving, at a central server, data from each of a plurality of data sources, the plurality of data sources being within a plurality of data storage locations, wherein the central server comprises a validation dataset having a plurality of annotated datapoints and wherein the central server trains a machine-learning model utilizing a training dataset” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“computing, updating, iteratively generating, at the central server” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:

"receiving, at a central server, data from each of a plurality of data sources, the plurality of data sources being within a plurality of data storage locations, wherein the central server comprises a validation dataset having a plurality of annotated datapoints and wherein the central server trains a machine-learning model utilizing a training dataset” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“computing, updating, iteratively generating, at the central server” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is subject-matter ineligible.

Regarding Claim 12:
The claim recites a system (“A computer program product, comprising”) that performs the method as described in claim 1. Therefore, claim 12 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 12 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“wherein the computer readable program code is configured to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))

Regarding Claims 2 and 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:  
“wherein the computing comprises evaluating the respective data of the data source against each annotated datapoint within the validation dataset” (a mental process, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None

Regarding Claims 3 and 14:
Subject Matter Eligibility Analysis Step 2A Prong 1:  
“wherein the computing comprises generating rankings of the plurality of data sources for each annotated datapoint based upon the evaluating” (a mental process, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None

Regarding Claims 4 and 15:
Subject Matter Eligibility Analysis Step 2A Prong 1: 
“wherein the computing comprises aggregating the rankings of the plurality of data sources across the plurality of annotated datapoints of the validation dataset” (a mental process, judgment)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: 

Regarding Claims 5 and 16:
Subject Matter Eligibility Analysis Step 2A Prong 1:  
“wherein the computing comprises computing an accuracy of the data source in predicting the annotations utilizing an influence function comprising a loss function component, a metrics of the data source component, and a gradient of the loss function component, wherein the loss function component and the gradient of the loss function component is computed at the central server and wherein a result of the metrics of the data source component is provided to the central server from a corresponding data source” (a mathematical calculation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None


Regarding Claims 6 and 17:
Subject Matter Eligibility Analysis Step 2A Prong 1: 
“further comprising constructing a bipartite graph comprising the plurality of annotated datapoints and the plurality of data sources by generating weighted edges between annotated datapoints and data sources based upon the influence of the data source” (a mental process, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None

Regarding Claims 7 and 18:
Subject Matter Eligibility Analysis Step 2A Prong 1:  
“further comprising selecting the subset of the plurality of data sources based upon a coverage budget provided by a user, wherein the coverage budget is selected from the group consisting of a first constraint on a number of the plurality of data sources and a second constraint on a number of the annotated datapoints” (a mental process, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None

Regarding Claims 8 and 19:
Subject Matter Eligibility Analysis Step 2A Prong 1:  
“further comprising selecting a least number of data sources that covers the validation dataset” (a mental process, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None

Regarding Claims 9 and 20:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: 
“wherein the respective data comprises a derivative of local data stored at the data source to preserve a privacy of the local data stored at the data source” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))

Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: 
“comprising training, at the central server, the machine-learning model utilizing the training dataset” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))


Regarding Claim 11:
The claim recites a system (“An apparatus, comprising”) that performs the method as described in claim 1. Therefore, claim 11 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 11 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“at least one processor” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“wherein the computer readable program code is configured to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))

Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.

Claims 1-4, 7, 10-11, 12-15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mars (US10296848B1), "Systems and Method for Automatically Configuring Machine Learning Models" in view of Krishnaswamy (US20220414464A1), "Method and Server for Federated Machine Learning".

Regarding claim 1, Mars teaches:
“A method, comprising: receiving, at a central server, data from each of a plurality of data sources, the plurality of data sources being within a plurality of data storage locations, wherein the central server comprises a validation dataset having a plurality of annotated datapoints and wherein the central server trains a machine-learning model utilizing a training dataset”([col. 2, lines 55-61;col. 5, lines 30-53; col. 10, lines 6-19; col. 11, lines 8-26; col. 13, lines 32-62, Figure 2], The ML management console (central server) collects training data from a plurality of external sources. The ML management console comprises a collection of seed samples (validation dataset) that can be used as input data to retrieve a plurality of labeled training samples from external data sources. After the console collects and process the ML training data, the training data is deployed to a ML model for execution. Slot identification engine may generate slot labels for user input data. It is implied that the seed samples are also labeled data because it is input data to the ML model and the seed samples are based on user query.)
“computing, at the central server, an influential score for each of the plurality of data sources based upon respective data provided to the central server from each of the plurality of data sources, wherein a respective influential score of a data source identifies an influence of the data source in accurately predicting annotations of the validation dataset when the respective data is used in the machine-learning model” ([col. 12, lines 15-58],”A first external training data source may have a high level of quality (judged based on a scale of 0-10, e.g., 8 level of quality or the like). A second external training data source may have a low level of quality (e.g., 2 level of quality, etc.)”, A level of quality is calculated and assigned to each of the plurality of the external training data sources and the quality of the data source relates to the accuracy (influential score) of labels generated by the data source. The quality score of a data source is based on the accuracy of the generated labels for the data samples.)
“updating, at the central server and based upon the influential score of the plurality of data sources, communication with the central server to a subset of the plurality of data ” ([col. 12, lines 15-58; col. 13, lines 5-31],” S250 applies the pruning threshold to the list of training data samples after the training data samples have been rated and/or ranked”, A threshold may be set for each of the external training data sources to limit the amount of data collected from the data source. After the fit score has been determine for the training data samples, the system may remove training data samples that have a fit score below a certain threshold.)
“iteratively generating, at the central server, the training dataset utilizing updated data received from the subset of the plurality of data sources” ([col. 13, lines 32-52], “load the training data samples collected from the plurality of external training data sources into one or more machine learning models of a machine 35 learning system”, After a subset of the data is determined from the external data sources, the system combines all the data into a training set and loads the training data samples into the ML model for execution. The system continuously received data samples to update and execute the ML model.)

Mars does not explicitly disclose an implementation of updating a subset of data sources. However, Krishnaswamy discloses in the same field of endeavor:
“updating, at the central server and based upon the influential score of the plurality of data sources, communication with the central server to a subset of the plurality of data sources” ([0096],”selecting the subset of data sources from the set of data sources (binned in the plurality of intervals) for federation”, The plurality of data sources may be categorized and grouped together based on different data qualities. The server may select a sub-set of data sources based on quality to generate data for training of the global ML model.)

It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of selecting a subset of data sources from Krishnaswamy into the teaching of Mars. Doing so can train a ML model based on different data quality parameters to improve the performance of the ML model prediction (Krishnaswamy, abstract).

Regarding claim 12, Mars teaches:
Claim 12 recites a system (“A computer program product, comprising”) that performs the same process as described in Claim 1. Therefore claim 12 is rejected under the same reasons mention for claim 1. However, claim 12 has additional limitations and the claim elements are addressed below:
“a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code executable by a processor” ([col. 14, lines 6-20],” a machine configured to receive a computer-readable medium storing computer-readable instructions. The computer-executable component is preferably a general or application specific processor”, A processor executes instructions (program code) from a computer-readable medium to perform the methods described by the reference.)
“wherein the computer readable program code is configured to” ([col. 14, lines 6-20],”The system and methods of the preferred embodiment and variations thereof can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions”, A plurality of computer-readable instructions is configured to perform the methods described by the reference.)

Regarding claims 2 and 13, Mars teaches:
“wherein the computing comprises evaluating the respective data of the data source against each annotated datapoint within the validation dataset”([col. 12, lines 36-58],”the fit score may be calculated based on the text of the training data samples matching or substantially matching a text or a meaning of a text of a seed sample”, The system processes the training data to determine a fit score (influential score) to rank each of the training data samples from external sources. The fit score describes how well a training data sample matches the seed samples (validation set) of a training request and the overall data quality in representing the ML model task.)

Regarding claims 3 and 14, Mars teaches:
“wherein the computing comprises generating rankings of the plurality of data sources for each annotated datapoint based upon the evaluating”([col. 11, lines 35-44; col. 12, lines 36-58],”S250 may function to rate and/or rank each of the training data samples from the external training data sources”, The system uses the calculated fit score to generate a ranking for each of the data samples that comes from the plurality of data sources. The ranking represents how valuable a particular data source is for training the ML model. The training data samples from various external data source can be stored into distinct datastores for specific processing of the set of data.)

Regarding claims 4 and 15, Mars teaches:
“wherein the computing comprises aggregating the rankings of the plurality of data sources across the plurality of annotated datapoints of the validation dataset”([col. 11, lines 35-44; col. 13, lines 1-4],”S250 may function to rank order each of the training data samples in descending or ascending ranking order”, The training data samples from various external data source can be stored into distinct datastores for specific processing of the set of data. The fit score may be calculated for a set of training samples that originated from the same data source. The system lists (aggregate) the rank based on the fit score in descending or ascending order.)


Regarding claims 7 and 18, Mars teaches:
“further comprising selecting the subset of the plurality of data sources based upon a coverage budget provided by a user, wherein the coverage budget is selected from a group consisting of a first constraint on a number of the plurality of data sources and a second constraint on a number of the annotated datapoints”([col. 10, lines 17-27; col. 12, lines 4-35], “An administrator may simply provide input identifying the one or more external training data sources. S240 may implement one or more thresholds for each of the plurality of external training data sources that function to limit an amount of training data. The limits or threshold for each of the plurality of external training data sources may be preset (e.g., may be an input value at the configuration console)”, A user interface is provided where the administrator (user) is able to provide a specific number (first constraint) of external training data sources from a list of external data sources. It is implied that the user may have prior knowledge to gather data from known data sources that can highly benefit the ML model training and limit the data gather from a selected list of external data sources. In the user interface, the administrator may also provide an input value to define the threshold for limiting the number of training data samples (second constraint) receive from the external data sources.)

Regarding claim 10, Mars teaches:
“comprising training, at the central server, the machine-learning model utilizing the training dataset” ([col. 13, lines 32-62], “load the training data samples collected from the plurality of external training data sources into one or more machine learning models of a machine 35 learning system”, After a subset of the data is determined from the external data sources, the system combines all the data into a training set and loads the training data samples into the ML model for execution.)

Regarding claim 11, Mars teaches:
Claim 11 recites a system (“An apparatus, comprising”) that performs the same process as described in Claim 1. Therefore claim 11 is rejected under the same reasons mention for claim 1. However, claim 11 has additional limitations and the claim elements are addressed below:
“at least one processor” ([col. 14, lines 6-20],”The computer-executable component is preferably a general or application specific processor”, A processor executes instructions (program code) from a computer-readable medium to perform the methods described by the reference.)
“a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor” ([col. 14, lines 6-20],” a machine configured to receive a computer-readable medium storing computer-readable instructions. The computer-executable component is preferably a general or application specific processor”, A processor executes instructions (program code) from a computer-readable medium to perform the methods described by the reference.)
“wherein the computer readable program code is configured to” ([col. 14, lines 6-20],”The system and methods of the preferred embodiment and variations thereof can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions”, A plurality of computer-readable instructions is configured to perform the methods described by the reference.)


Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mars (US10296848B1), "Systems and Method for Automatically Configuring Machine Learning Models" in view of Krishnaswamy (US20220414464A1), "Method and Server for Federated Machine Learning" and Hall (US20230162049A1), "Artificial Intelligence (AI) Method for Cleaning Data for Training AI Models".

Regarding claims 5 and 16, Mars in view of Krishnaswamy teaches:
“wherein the computing comprises computing an accuracy of the data source in predicting the annotations utilizing an influence function Mars teaches the system (central server) may compute an operational metric (accuracy) of the ML model in making accurate predictions or classifying labels accurately. The system retrieves training data samples from external data sources. The system validates the performance of the ML model based on the training data samples by determining and comparing operational metrics to determine the reliability of the external data sources. The computation of the operational metrics is performed in a central server system after receiving the training data samples from external data sources. Krishnaswamy further teaches reliability metrics to score the quality of the labelled data within a data source. The data quality indices may be computed locally at the data source and the results are sent back to the main server.)

Mars in view of Krishnaswamy does not explicitly disclose an implementation of a computing function “comprising a loss function component, and a gradient of the loss function component”. However, Hall discloses in the same field of endeavor:
“wherein the computing comprises computing an accuracy of the data source in predicting the annotations utilizing an influence function comprising a loss function component, a metrics of the data source component, and a gradient of the loss function component, wherein the loss function component and the gradient of the loss function component is computed at the central server and wherein a result of the metrics of the data source component is provided to the central server from a corresponding data source” ([0169, 0185-0187],”the balanced accuracy metric is used rather than overall accuracy because in some cases the skewed class distribution on a dataset can be associated with very high overall accuracy even though the balanced accuracy is only around 50% (see an example of this below in the experimental results section). However other metrics may be used including confidence metrics such as Log Loss”, The training data for the ML model may come from a variety of data sources. In one embodiment, data may be collected at a central server to perform the method of determining the predictive power of the dataset from a particular data source. A balanced accuracy metric or a log loss function may be used to evaluate the dataset for each data source to determine the accuracy and data quality of a dataset.)

It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a computing function “comprising a loss function component, and a gradient of the loss function component” from Hall into the teaching of Mars in view of Krishnaswamy. Doing so can train a global ML model with high quality data and remove the noisy data in the dataset using metrics to determine which data are consistently providing incorrect predictions (Hall, abstract).

Claims 6 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Mars (US10296848B1), "Systems and Method for Automatically Configuring Machine Learning Models" in view of Krishnaswamy (US20220414464A1), "Method and Server for Federated Machine Learning" and Rounthwaite (US20100325133A1), "Determining a Similarity Measure Between Queries".

Regarding claims 6 and 17, Mars in view of Krishnaswamy teaches:
“further comprising The aggregator server may compute a federated weights of the plurality of data quality parameters for each of the data source. An example of a data quality parameter is label quality that evaluates the annotation method used to produce the labelled data in the data source. A relationship may be determined between each data source and the quality of labelled data produced by the data store.)

Mars in view of Krishnaswamy does not explicitly disclose an implementation of representing the data quality as a bipartite graph. However, Rounthwaite discloses in the same field of endeavor:
“wherein the selecting comprises constructing a bipartite graph comprising the plurality of ” ([0021],” the bipartite graph includes a first plurality of nodes and a second plurality of nodes, and wherein nodes in the first plurality of nodes can be coupled to nodes in the second plurality of nodes by edges. The edge can be weighted”, A bipartite graph is generated to show a visual representation of the relationship between two types of entities. A weighted edge defines how relevant a first set of nodes map to a second set of nodes.)

It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of representing the data quality as a bipartite graph from Rounthwaite into the teaching of Mars in view of Krishnaswamy. Doing so can use a visual graphical representation to define high correlations between 2 different sets of data (Rounthwaite, par. 21).

Claims 8 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Mars (US10296848B1), "Systems and Method for Automatically Configuring Machine Learning Models" in view of Krishnaswamy (US20220414464A1), "Method and Server for Federated Machine Learning" and McDonald (US20170212241A1), "Using Space Based Augmentation System (SBAS) Ephemeris Sigma Information to Reduce Ground Based Augmentation Systems (GBAS) Ephemeris Decorrelation Parameter".

Regarding claims 8 and 19, Mars in view of Krishnaswamy teaches:
“further comprising selecting N)”, Mars teaches a training data seed samples (validation dataset) is a set of dataset that a user would like to obtain more training data that are similar to the seed samples from a plurality of external data sources. Krishnaswamy teaches a selection of a subset of data sources.)

Mars in view of Krishnaswamy does not explicitly disclose an implementation of selecting the least number of data sources. 
However, McDonald discloses in the same field of endeavor:
“wherein the selecting comprises selecting the least number of data sources that covers the The system evaluate the data from multiple data sources and selects a data source that meets a certain criterion, such as a data quality metric.)

It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of selecting the least number of data sources from McDonald into the teaching of Mars in view of Krishnaswamy. Doing so can select the data with the lowest uncertainty value to obtain the best results (Thompson, par. 33).

Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mars (US10296848B1), "Systems and Method for Automatically Configuring Machine Learning Models" in view of Krishnaswamy (US20220414464A1), "Method and Server for Federated Machine Learning" and Choudhury (US20210150269A1), " Anonymizing Data for Preserving Privacy During use for Federated Machine Learning".

Regarding claims 9 and 20, Mars in view of Krishnaswamy, teaches: 
“wherein the respective data comprises a ” ([0003, 0081],” In federated learning, training data may be retained at their native locations (data sources)”, A local ML model is used to train the ML model using private data located within a data source. The training of the ML model at the local data source preserves the privacy of the data because sensitive information is not shared with servers outside of the data source.)
Mars in view of Krishnaswamy does not explicitly disclose an implementation of storing a derivative of the data. However, Choudhury discloses in the same field of endeavor:
“wherein the respective data comprises a derivative of local data stored at the data source to preserve a privacy of the local data stored at the data source” ([0039-0040],” as part of privacy-protecting the training data D1 before using it to train the federated model 112, the method 200 includes enabling the local nodes to decide on which attributes 320 to use to create and train the federated model 112”, The training data at each local data source have certain attributes filtered out prior to training the federated learning model. Attributes like direct identifiers are not used to train the model.)

It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of storing a derivative of the data from Choudhury into the teaching of Mars. Doing so can protect sensitive information from being leaked by a data source during training of a global federated learning model (Choudhury, abstract).

Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.



/GARY MAC/Examiner, Art Unit 2127                                                                                                                                                                                                        
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127                                                                                                                                                                                                        


    
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
        
            
    


Cookies help us deliver our services. By using our services, you agree to our use of cookies.