Patent Application 17928976 - CLASSIFICATION OF BLOOD CELLS - Rejection
Appearance
Patent Application 17928976 - CLASSIFICATION OF BLOOD CELLS
Title: CLASSIFICATION OF BLOOD CELLS
Application Information
- Invention Title: CLASSIFICATION OF BLOOD CELLS
- Application Number: 17928976
- Submission Date: 2025-05-14T00:00:00.000Z
- Effective Filing Date: 2022-12-01T00:00:00.000Z
- Filing Date: 2022-12-01T00:00:00.000Z
- National Class: 382
- National Sub-Class: 134000
- Examiner Employee Number: 90927
- Art Unit: 2669
- Tech Center: 2600
Rejection Summary
- 102 Rejections: 1
- 103 Rejections: 14
Cited Patents
The following patents were cited in the rejection:
- US 0227495đ
- US 0216745đ
Office Action Text
DETAILED ACTION Response to Amendment The preliminary amendment filed December 1, 2022, has been entered in full. Claims 1-24 are pending. Information Disclosure Statement The information disclosure statements (IDS) submitted on December 1, 2022; April 2, 2023; and July 25, 2024, are being considered by the examiner. Claim Objections Claim(s) 4, 18, and 24 is/are objected to because of the following informalities: In claim 4, second-to-last line âeach identified centroidsâ should be âeach identified centroidâ (i.e., delete trailing âsâ) In claim 18, second-to-last line âeach identified centroidsâ should be âeach identified centroidâ (i.e., delete trailing âsâ) In claim 24, line 7, âgenerating cell images magesâ should be âgenerating cell imagesâ (i.e., delete âmagesâ) Appropriate correction is required. Claim Interpretation The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. â An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term âmeansâ or âstepâ or a term used as a substitute for âmeansâ that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term âmeansâ or âstepâ or the generic placeholder is modified by functional language, typically, but not always linked by the transition word âforâ (e.g., âmeans forâ) or another linking word or phrase, such as âconfigured toâ or âso thatâ; and (C) the term âmeansâ or âstepâ or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word âmeansâ (or âstepâ) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word âmeansâ (or âstepâ) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word âmeansâ (or âstepâ) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word âmeansâ (or âstepâ) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word âmeans,â but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: the âblood monitoring deviceâ in claim 1. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.âThe specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claim(s) 5, 6, 10, 15, and 17 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 5 recites the limitation "the respective class of sickle red blood cells (sRBCs)" in line 3. There is insufficient antecedent basis for this limitation in the claim. Claim 15 is also indefinite for substantially the same reason as claim 5. Claims 6 and 17 are also indefinite at least because they include the indefinite limitations of claims 5 and 15, respectively. Claim 10 recites the limitation "the type of blood cell" in line 4. There is insufficient antecedent basis for this limitation in the claim. There is no prior recitation of a type of blood cell in claim 10, or in claims 9 and 1, from which it depends. Claim Rejections - 35 USC § 102 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless â (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claim(s) 12, 13, 16, and 24 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by âZhangâ (âBlood Cell Image Classification Based on Image Segmentation Preprocessing and CapsNet Network Model,â 2019). Regarding claim 12, Zhang discloses a system (e.g., Section 2.1.2; Table I) comprising: a processor (e.g., Table I, GPU or CPU); one or more non-transitory machine readable media to store instructions and data (e.g., Table I, RAM and/or hard disk), the data including an image of a blood sample (e.g., Sec. 2.1.1, blood image data; Fig. 2 shows further examples), the processor configured to access the media and execute the instructions (e.g., Sec. 1, last paragraph) comprising: a machine learning model (e.g., Fig. 3, Sec. 2.2, U-Net) trained to generate a segmentation mask (e.g., Fig. 3, right, lower output; Fig. 4 has additional examples) that assigns pixels in the image to one of a plurality of classes (e.g., Fig. 3, right, lower output, pixels are assigned to either a background/non-interest class masked in black or a white blood cell class) that correlate to respective known biophysical properties of blood cells (e.g., Fig. 1, it is known that blood includes white blood cells and other constituents, which have various known biophysical properties); and extraction code programmed to extract cell images from the input image based on the segmentation mask, in which each extracted cell image includes a respective cluster of the pixels assigned to a respective one of the plurality of classes (e.g., Fig. 3, right, lower output shows an extracted cell image formed by applying the segmentation mask to the input image, which results in non-cell regions being marked black and a cluster of pixels assigned to the white blood cell class being shown in their original color from the input image â best seen in a color copy of the reference; Also see Fig. 7, 32x32 input image). Regarding claim 13, Zhang discloses the system of claim 12, wherein the machine learning model is a first machine learning model trained to detect a type of blood cell (e.g., Sec. 2.2.2, 1st par., the 1st segmentation machine learning model is trained to detect white blood cells) that includes more than one of the plurality of classes (e.g., Fig. 1, white blood cells include more than one class, such as Eosinophil, Lymphocyte, Monocyte, and Neutrophil classes), the instructions further comprising a second machine learning model (e.g., Sec. 2.3, Fig. 7, CapsNet) trained to classify each of the extracted cell images to specify morphological subtypes for the type of blood cell detected by the first machine learning model (e.g., Sec. 1, last par., âclassification of four major types of leukocytes (eosinophils, lymphocytes, monocytes, neutrophils)â â i.e., four morphological subtypes of white blood cells; e.g., Fig. 7, top-right, 4-class output). Regarding claim 16, Zhang discloses the system of claim 13, wherein the second machine learning model comprises a second convolutional neural network to employ convolutions and filters to classify each of the cell images (e.g., Fig. 7, Sec. 2.3.2 (2), CapsNet is a convolutional neural network including various filter kernels that are convolved with the input image and/or feature maps to classify each of the cell images). Regarding claim 24, Zhang discloses one or more non-transitory machine readable media having instructions, executable by a processor to perform a method (e.g., Table I, RAM and/or hard disk; Sec. 1, last par.) comprising: retrieving image data that includes an input image of a blood sample (e.g., Sec. 2.1.1, blood image data; Fig. 2 shows further examples); using a first neural network (e.g., Fig. 3, Sec. 2.2, U-Net) to generate a segmentation mask (e.g., Fig. 3, right, lower output; Fig. 4 has additional examples) that assigns each pixel in the image to a respective class of a plurality of classes (e.g., Fig. 3, right, lower output, pixels are assigned to either a background/non-interest class masked in black or a white blood cell class) that correlate to respective known biophysical properties (e.g., Fig. 1, it is known that blood includes white blood cells and other constituents, which have various known biophysical properties); generating cell images mages from the input image based on the segmentation mask in which each cell image includes a respective cluster of the pixels assigned to a respective one of the plurality of classes (e.g., Fig. 3, right, lower output shows an extracted cell image formed by applying the segmentation mask to the input image, which results in non-cell regions being marked black and a cluster of pixels assigned to the white blood cell class being shown in their original color from the input image â best seen in a color copy of the reference; Also see Fig. 7, 32x32 input image); and providing the input image set to a second neural network (e.g., Fig. 7, CapsNet) to classify respective objects in the input image set as corresponding to one or more subclasses of the respective class (e.g., Sec. 1, last par., âclassification of four major types of leukocytes (eosinophils, lymphocytes, monocytes, neutrophils)â â i.e., cell objects in input image are classified as one of four subclasses of white blood cells; e.g., Fig. 7, top-right, 4-class output). Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim(s) 14, 15, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of âXuâ (âA deep convolutional neural network for classification of red blood cells in sickle cell anemia,â 2017). Regarding claim 14, Zhang teaches the system of claim 12. Zhang uses machine learning techniques to detect white blood cells in images of blood samples (e.g., Figs. 3 and 7) and teaches that the plurality of classes includes background (i.e., âotherâ) and white blood cells (e.g., Fig. 3, lower output, and Fig. 4, third and sixth rows, pixels are either background shown as black in the masked output or white blood cells, which are shown as colored, non-masked regions). Zhang does not teach, in addition to background, classes of deformable adhered sickle red blood cell (sRBC), non-deformable adhered sRBC, and non-functionally adhered/other deformable sRBC. However, Xu does teach using machine learning techniques to detect red blood cells in images of blood samples (e.g., Page 9, RBC pattern classification based on deep CNN), where classes of red blood cells include at least other deformable sRBC (e.g., Table 1, no. 7, sickle red blood cells; Page 20, Classification of deoxygenated sickle RBCs, 2nd par., the sickle red blood cells are deformable at least due to oxygenation/deoxygenation; Also see Fig. 18). Xuâs teachings indicate that, as with white blood cells, there is a need to automatically identify sickle red blood cells (e.g., Introduction, 2nd par., âimplementing an automated, high-throughput cell classification method could become an enabling technology to improve the future clinical diagnosis, prediction of treatment outcome, and especially therapy planningâ), demonstrate that machine learning techniques can provide such automatic identification (e.g., Page 24, last par.), and suggest that its techniques could be applied to other types of blood cells (Page 25, 2nd and 3rd pars.). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the other deformable sRBC classification of Xu in order to improve the method with the reasonable expectation that this would result in a method that could identify other types of clinically-relevant blood cells, thereby advantageously providing additional useful diagnostic information. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Xu. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Xu to obtain the invention as specified in claim 14. Regarding claim 15, Zhang teaches the system of claim 13. Zhang further teaches that the first machine learning model comprises a first convolutional neural network to classify pixels in the image according to the respective class (e.g., Fig. 3, output, pixels are classified as being background/other class marked as black or white blood cell shown in color) and to generate the segmentation mask to include cell objects segmented from clusters of pixels based on the assigned class of the pixels (e.g., Fig. 3, output, segmentation mask shown in lower output includes a cell object that is a cluster of pixels within the boundary shown as a red outline in the upper output based on the assigned class of white blood cell pixels). As explained above, âthe respective class of sickle red blood cells (sRBCs)â is an indefinite limitation. As best understood in view of the indefiniteness, it is being interpreted to require that at least one of the plurality of classes recited in claim 12 is an sRBC class. Zhang uses machine learning techniques to detect white blood cells in images of blood samples (e.g., Figs. 3 and 7) and teaches that the plurality of classes includes background (i.e., âotherâ) and white blood cells (e.g., Fig. 3, lower output, and Fig. 4, third and sixth rows, pixels are either background shown as black in the masked output or white blood cells, which are shown as colored, non-masked regions). Zhang does not teach a sickle red blood cell (sRBC) class. However, Xu does teach using machine learning techniques to detect red blood cells in images of blood samples (e.g., Page 9, RBC pattern classification based on deep CNN), where classes of red blood cells include at least an sRBC class (e.g., Table 1, no. 7, sickle red blood cells; Page 20, Classification of deoxygenated sickle RBCs, the sRBCs may have oxygenated and deoxygenated subtypes; Also see Fig. 18). Xuâs teachings indicate that, as with white blood cells, there is a need to automatically identify sickle red blood cells (e.g., Introduction, 2nd par., âimplementing an automated, high-throughput cell classification method could become an enabling technology to improve the future clinical diagnosis, prediction of treatment outcome, and especially therapy planningâ), demonstrate that machine learning techniques can provide such automatic identification (e.g., Page 24, last par.), and suggest that its techniques could be applied to other types of blood cells (Page 25, 2nd and 3rd pars.). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the sRBC classification of Xu in order to improve the method with the reasonable expectation that this would result in a method that could identify other types of clinically-relevant blood cells, thereby advantageously providing additional useful diagnostic information. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Xu. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Xu to obtain the invention as specified in claim 15. Regarding claim 17, Zhang in view of Xu teaches the system of claim 15, and Zhang further teaches that the first convolutional neural network is programmed to downsample the input image to generate feature vectors (e.g., Fig. 3, U-Net, image is downsampled to generate various feature vectors on the left side) and to upsample based on the feature vectors to generate the segmentation mask (e.g., Fig. 3, U-Net, feature vectors are upsampled to generate output segmentation mask on the right side). Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of âXiaâ (âAutomated Blood Cell Detection and Counting via Deep Learning for Microfluidic Point-of-Care Medical Devices,â 2019). Regarding claim 18, Zhang teaches the system of claim 13. Zhang teaches techniques for automatically detecting white blood cells in blood sample images (e.g., Figs. 3 and 7). Zhang recognizes that â[t]he composition of white blood cells in blood reveals important diagnostic information for patientsâ (Sec. 1, 1st par.), but does not explicitly teach counting white blood cells to determine that composition. However, Xia also teaches techniques for automatically detecting white blood cells in blood sample images (e.g., Figs. 3 and 4) and further suggests using the detections for automatic white blood cell counting (e.g., Sec. 5). Xia teaches that white blood cell counting is âa routine testâ and can be used to various diagnoses (Page 2, below Fig. 1). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the blood cell counting of Xia in order to improve the system with the reasonable expectation that this would result in a system that advantageously produced routine test results useful for diagnosis. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Xia. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Xia to obtain the invention as specified in claim 18. Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of âGildenblatâ (US 2021/0216745 A1). Regarding claim 19, Zhang teaches the system of claim 12, and Zhang further teaches that the extraction code is further programmed to: cluster the pixels in the segmentation mask to define respective pixel clusters according to the assigned class to detect cell objects in the input image (e.g., Fig. 3, outputs, segmentation mask is applied to remove areas of non-interest, leaving a cluster of pixels according to the assigned class of white blood cells, thereby detecting objects in the input image; Also see Fig. 4, third and sixth rows). Zhang applies U-net to obtain a mask from a 128x128-pixel input image (Fig. 3). The mask is used to extract a 32x32-pixel cell image for input to the classifier (e.g., Fig. 7, input; Table II, top row), but does not explicitly teach how the 32x32-pixel image is formed. In particular, Zhang does not explicitly teach identifying centroids for each respective pixel cluster; and detecting bounding boxes around each identified centroid, wherein the each cell image is generated based on pixels within the detected bounding box. However, Gildenblat does teach a similar technique (e.g., Fig. 4) where U-Net is used to detect cells (e.g., Fig. 4, C2; Figs. 2-3 describe details of using U-Net for cell detection) and images of the detected cells are extracted for classification (e.g., Fig. 4, C3 and C6). The image extraction is performed by identifying centroids for each respective cell pixel cluster (e.g., [0039], Fig. 4, centers of cells are identified), and detecting 32x32-pixel bounding boxes around each identified centroid (e.g., [0039], Fig. 4, 32x32-pixel box is placed around each centroid), wherein each cell image is generated based on pixels within the detected bounding box (e.g., [0039], fig. 4, each 32x32-pixel box is cropped, thereby extracting pixels within the box as a cell image). The cropping technique of Gildenblat advantageously provides cell images of the 32x32-pixel size required by Zhang. It is also advantageous because the cropping removes extraneous pixels that may confuse a classifier and/or waste classification computations on irrelevant image sections. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the cell image cropping of Gildenblat in order to improve system with the reasonable expectation that this would result in a system that could produce the necessary 32x32-pixel cell images and did so in a way that removed extraneous pixels, thereby avoiding classifier confusion and/or wasted computations. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Gildenblat. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Gildenblat to obtain the invention as specified in claim 19. Claim(s) 20-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang as applied to claim 12 above, and further in view of âGurkanâ (US 2017/0227495 A1). Regarding claim 20, Zhang teaches the system of claim 13. Zhang automatically detects white blood cells in a blood sample (e.g., Figs. 3 and 7), but does not teach a microfluidic device to contain the blood sample, the microfluidic device comprising a channel having at least one functionalized adhesion region adapted to adhere to blood cells of interest within the blood sample. However, Gurkan does teach a microfluidic device to contain the blood sample (e.g., Fig. 8B, SCD Biochip), the microfluidic device comprising a channel having at least one functionalized adhesion region adapted to adhere to blood cells of interest within the blood sample (e.g., Fig. 8B, [0096]-[0098], the SCD Biochip includes multiple channels with functionalized adhesion regions adapted to adhere to blood cells of interest within the blood sample, including one for sicked red blood cells (RBCs) and one for white blood cells (WBCs)). Gurkan teaches that sickle cell disease (SCD) can be associated with abnormal adhesion of red and white blood cells (RBCs and WBCs) (e.g., [0075]-[0076]). Gurkan further teaches traditional blood tests for SCD take too long (e.g., [0004]) and that its microfluidic device blood sampling solves this problem (e.g., [0121]). Gurkan also suggests using image processing to identify and quantify cells in its microfluidic blood sample device (e.g., [0097], [0116]). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the microfluidic device of Gurkan in order to improve the system with the reasonable expectation that this would result in a system that could provide faster blood testing necessary for diagnosis and treatment of sickle cell disease. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Gurkan. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Gurkan to obtain the invention as specified in claim 20. Regarding claim 21, Zhang in view of Gurkan teaches the system of claim 20. As explained above, Zhang focuses on detecting white blood cells in blood samples (e.g., Sec. 1, last par.) and Gurkan teaches a microfluidic device that captures not only adhered white blood cells, but also adhered sickle red blood cells sRBCs (e.g., Fig. 8B). Therefore, while Gurkanâs teachings demonstrate that at least some of the known biophysical properties of white blood cells relate to an adhesive property of blood cells within the adhesion region (e.g., Fig. 8B, [0096]-[0098]), Zhang does not explicitly teach detecting (see claim 13) a type of blood cell that includes a combined class of adhered sickle red blood cells (sRBCs). However, Gurkan does teach using a single imaging system to quantify adhered cells in each of the channels of the biochip (e.g., [0116]), which include channels with both adhered white blood cells and adhered sickle red blood cells (sRBCs) (e.g., Fig. 8B, [0096]-[0098]). Zhang also acknowledges prior art image processing that was applied to both red and white blood cells (e.g., Sec. 1, 3rd par.). One of ordinary skill in the art would also recognize that the U-Net and CapsNet used by Zhang are general image processing tools that are not specific or limited to white blood cell images. Taken together, this information would suggest to one of ordinary skill in the art that the blood cell segmentation techniques of Zhang could be applied to detect not only white blood cells, but also red blood cells, such as the adhered sRBCs that are also captured by the microfluidic biochip of Gurkan, with a reasonable expectation of success. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang in view of Gurkan as applied above to additionally apply the automatic cell detection techniques of Zhang to the adhered sRBC cells additionally captured by Gurkanâs microfluidic device in order to improve the system with the reasonable expectation that this would result in a system that produced additional diagnostic information useful for the treatment of sickle cell disease. This technique for improving the system of Zhang in view of Gurkan was within the ordinary ability of one of ordinary skill in the art based on the teachings of Zhang and Gurkan and the knowledge generally available to one of ordinary skill in the art. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Gurkan to obtain the invention as specified in claim 21. Regarding claim 22, Zhang in view of Gurkan teaches the system of claim 20, and Gurkan further teaches that the image is acquired while the blood is flowing through the channel of the microfluidic device (e.g., [0180], imaging my microscope; e.g., [0182], condition (2) flow). Regarding claim 23, Zhang in view of Gurkan teaches the system of claim 20, and Gurkan further teaches that the image is acquired while the blood is not flowing through the microfluidic device (e.g., [0180], imaging my microscope; e.g., [0182], condition (1) no flow). Claim(s) 1-3 and 8-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan. Regarding claim 1, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 12, except that it additionally requires the blood sample to be âwithin a blood monitoring deviceâ. As explained above (see Claim Interpretation), the blood monitoring device is interpreted under 35 U.S.C. 112(f). Its corresponding structure includes a microfluidic device â see, e.g., par. [0050] and Fig. 3, microfluidic device 152, of the published application. Zhang teaches the system of claim 12 (see above), but does not teach a microfluidic device to contain the blood sample. However, Gurkan does teach a microfluidic device to contain a blood sample (e.g., [0096]-[0098], Fig. 8B, SCD Biochip). Gurkan teaches that sickle cell disease (SCD) can be associated with abnormal adhesion of red and white blood cells (RBCs and WBCs) (e.g., [0075]-[0076]). Gurkan further teaches traditional blood tests for SCD take too long (e.g., [0004]) and that its microfluidic device blood sampling solves this problem (e.g., [0121]). Gurkan also suggests using image processing to identify and quantify cells in its microfluidic blood sample device (e.g., [0097], [0116]). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang with the microfluidic device of Gurkan in order to improve the system with the reasonable expectation that this would result in a method that could provide faster blood testing necessary for diagnosis and treatment of sickle cell disease. This technique for improving the method of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Gurkan. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Gurkan to obtain the invention as specified in claim 1. Regarding claim 2, Examiner notes that the claim depends from claim 1 and recites limitations that are substantially the same as limitations recited in claim 13. The invention of claim 1 is obvious over Zhang in view of Gurkan (see above) and Zhang discloses the limitations of claim 13 (see above). Accordingly, claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan for substantially the same reasons presented in the rejections of claims 1 and 13. Regarding claim 3, Examiner notes that the claim depends from claim 2 and recites limitations that are substantially the same as limitations recited in claim 16. The invention of claim 2 is obvious over Zhang in view of Gurkan (see above) and Zhang discloses the limitations of claim 16 (see above). Accordingly, claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan for substantially the same reasons presented in the rejections of claims 2 and 16. Regarding claim 8, Zhang in view of Gurkan teaches the method of claim 1. Zhang uses machine learning techniques to detect white blood cells in images of blood samples (e.g., Figs. 3 and 7) and teaches that the plurality of classes includes background (i.e., âotherâ) and white blood cells (e.g., Fig. 3, lower output, and Fig. 4, third and sixth rows, pixels are either background shown as black in the masked output or white blood cells, which are shown as colored, non-masked regions). Zhang does not teach deformable adhered sickle red blood cell (sRBC), non-deformable adhered sRBC, and non-functionally adhered/other deformable sRBC classes. As explained above, Gurkan teaches a microfluidic device that captures not only adhered white blood cells, but also adhered sickle red blood cells sRBCs (e.g., [0096]-[0098], Fig. 8B). The adhered sRBCs may be deformable or non-deformable (e.g., [0182]). Gurkan teaches using a single imaging system to quantify adhered cells in each of the channels of the biochip (e.g., [0116]), which include channels with both adhered white blood cells and adhered sickle red blood cells (sRBCs) (e.g., Fig. 8B, [0096]-[0098]). Zhang also acknowledges prior art image processing that was applied to both red and white blood cells (e.g., Sec. 1, 3rd par.). One of ordinary skill in the art would also recognize that the U-Net and CapsNet used by Zhang are general image processing tools that are not specific or limited to white blood cell images. Taken together, this information would suggest to one of ordinary skill in the art that the blood cell segmentation techniques of Zhang could be applied to detect not only white blood cells, but also red blood cells, such as the adhered sRBCs that are also captured by the microfluidic biochip of Gurkan, with a reasonable expectation of success. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang in view of Gurkan as applied above to additionally apply the automatic cell detection techniques of Zhang to the deformable and/or non-deformable adhered sRBCs additionally captured by Gurkanâs microfluidic device in order to improve the system with the reasonable expectation that this would result in a system that produced additional diagnostic information useful for the treatment of sickle cell disease. This technique for improving the system of Zhang in view of Gurkan was within the ordinary ability of one of ordinary skill in the art based on the teachings of Zhang and Gurkan and the knowledge generally available to one of ordinary skill in the art. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Gurkan to obtain the invention as specified in claim 8. Regarding claim 9, Examiner notes that the claim depends from claim 1 and recites limitations that are substantially the same as limitations recited in claim 20. The invention of claim 1 is obvious over Zhang in view of Gurkan (see above) and Zhang in view of Gurkan teaches the limitations of claim 20 (see below). Accordingly, claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan for substantially the same reasons presented in the rejections of claims 1 and 20. Regarding claim 10, Examiner notes that the claim depends from claim 9 and recites limitations that are substantially the same as limitations recited in claim 21. The invention of claim 10 is obvious over Zhang in view of Gurkan (see above) and Zhang in view of Gurkan teaches the limitations of claim 21 (see below). Accordingly, claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan for substantially the same reasons presented in the rejections of claims 9 and 21. Regarding claim 11, Examiner notes that the claim recites one or more non-transitory machine readable media having instructions, which when executed by a processor perform the method of claim 1. Zhang in view of Gurkan teaches the method of claim 1 (see above). Zhang further teaches implementation of the method using one or more non-transitory machine readable media having instructions, which when executed by a processor perform the method (e.g., Table I, RAM and/or hard disk; Sec. 1, last par.). Accordingly, claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan for substantially the same reasons presented in the rejection of claim 1. Claim(s) 4-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan as applied to claim 1, and further in view of Gildenblat. Regarding claim 4, Examiner notes that the claim depends from claim 1 and recites limitations that are substantially the same as limitations recited in claim 19. The invention of claim 1 is obvious over Zhang in view of Gurkan (see above) and Zhang in view of Gildenblat teaches the limitations of claim 19 (see above). Accordingly, claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan and further in view of Gildenblat for substantially the same reasons presented in the rejections of claims 1 and 19. Regarding claim 5, Zhang in view of Gurkan and Gildenblat teaches the method of claim 4. Zhang further teaches that the first machine learning model comprises a first convolutional neural network to classify pixels in the image according to the respective class (e.g., Fig. 3, output, pixels are classified as being background/other class marked as black or white blood cell shown in color) and to generate the segmentation mask to include cell objects segmented from clusters of pixels based on the assigned class of the pixels (e.g., Fig. 3, output, segmentation mask shown in lower output includes a cell object that is a cluster of pixels within the boundary shown as a red outline in the upper output based on the assigned class of white blood cell pixels). As explained above, âthe respective class of sickle red blood cells (sRBCs)â is an indefinite limitation. As best understood in view of the indefiniteness, it is being interpreted to require that at least one of the plurality of classes recited in claim 12 is an sRBC class. Zhang uses machine learning techniques to detect white blood cells in images of blood samples (e.g., Figs. 3 and 7) and teaches that the plurality of classes includes background (i.e., âotherâ) and white blood cells (e.g., Fig. 3, lower output, and Fig. 4, third and sixth rows, pixels are either background shown as black in the masked output or white blood cells, which are shown as colored, non-masked regions). Zhang does not teach a sickle red blood cell (sRBC) class. As explained above, Gurkan teaches a microfluidic device that captures not only adhered white blood cells, but also adhered sickle red blood cells sRBCs (e.g., [0096]-[0098], Fig. 8B). Gurkan teaches using a single imaging system to quantify adhered cells in each of the channels of the biochip (e.g., [0116]), which include channels with both adhered white blood cells and adhered sickle red blood cells (sRBCs) (e.g., Fig. 8B, [0096]-[0098]). Zhang also acknowledges prior art image processing that was applied to both red and white blood cells (e.g., Sec. 1, 3rd par.). One of ordinary skill in the art would also recognize that the U-Net and CapsNet used by Zhang are general image processing tools that are not specific or limited to white blood cell images. Taken together, this information would suggest to one of ordinary skill in the art that the blood cell segmentation techniques of Zhang could be applied to detect not only white blood cells, but also red blood cells, such as the adhered sRBCs that are also captured by the microfluidic biochip of Gurkan, with a reasonable expectation of success. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang in view of Gurkan and Gildenblat as applied above to additionally apply the automatic cell detection techniques of Zhang to the adhered sRBC cells additionally captured by Gurkanâs microfluidic device in order to improve the system with the reasonable expectation that this would result in a system that produced additional diagnostic information useful for the treatment of sickle cell disease. This technique for improving the system of Zhang in view of Gurkan and Gildenblat was within the ordinary ability of one of ordinary skill in the art based on the teachings of Zhang and Gurkan and the knowledge generally available to one of ordinary skill in the art. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang, Gurkan, and Gildenblat to obtain the invention as specified in claim 5. Regarding claim 6, Zhang in view of Gurkan and Gildenblat teaches the method of claim 5, and Zhang further teaches that the first convolutional neural network is configured to downsample the input image to generate feature vectors (e.g., Fig. 3, U-Net, image is downsampled to generate various feature vectors on the left side) and to upsample based on the feature vectors to generate the segmentation mask (e.g., Fig. 3, U-Net, feature vectors are upsampled to generate output segmentation mask on the right side). Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan as applied to claim 2 above, and further in view of Xia. Regarding claim 7, Examiner notes that the claim depends from claim 2 and recites limitations that are substantially the same as limitations recited in claim 18. The invention of claim 2 is obvious over Zhang in view of Gurkan (see above) and Zhang in view of Xia teaches the limitations of claim 18 (see above). Accordingly, claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gurkan and further in view of Xia for substantially the same reasons presented in the rejections of claims 2 and 18. Conclusion The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Examiner notes that the âXiaâ reference is relied upon in the rejections above to teach counting blood cells, but it also includes further pertinent disclosures: Sampling blood in a microfluidic device â e.g., Fig. 1 A Faster R-CNN cell detection framework (Fig. 3) that includes A region proposal network (RPN) that extracts bounded sub-images corresponding to cells A CNN classifier that determines classes of the cells in the sub-images âKajĂĄnekâ (âEvaluation of Detection of Red Blood Cells using Convolutional Neural Networks,â 2019) Uses different CNN-based approaches, such as a custom sliding window classifier (Fig. 5) and Faster R-CNN (e.g., Sec. V, 5th par.), for detecting red blood cells (RBCs) in images of blood samples in microfluidic devices âHortinelaâ (âIdentification of Abnormal Red Blood Cells and Diagnosing Specific Types of Anemia Using Image Processing and Support Vector Machine,â 2019) Applies heuristic (i.e., non-machine-learning) segmentation (e.g., Figs. 8-9) and SVM classification (e.g., Figs. 8 and 10) to blood sample images to classify red blood cells (RBCs) according to their morphology (e.g., Table III) âde Haanâ (âAutomated screening of sickle cells using a smartphone-based microscope and deep learning,â 22 May 2020) Performs semantic segmentation (i.e., simultaneous segmentation and classification) on sickle RBCs â e.g., Fig. 1 âDhiebâ (âAn Automated Blood Cells Counting and Classification Framework using Mask R-CNN Deep Learning Model,â 2019) Uses Mask R-CNN to detect both red and white blood cells in a blood sample â Figs. 1 and 3 âRazzakâ (âMicroscopic Blood Smear Segmentation and Classification using Deep Contour Aware CNN and Extreme Learning Machine,â 2017) Uses a contour-aware neural network to segment cells in a blood sample image and extract cell images â Sec. 3.1; Fig. 2, Context Awared FCN Uses an extreme learning machine to classify each blood cell by type â Sec. 3.2; Fig. 2, ELM Classifier-I RBCs are further classified into different subtypes including sickle cells â Sec. 3.2; Fig. 2, Classifier-II âTriscottâ (âA Simple Microfluidic Device for Automated, High-Throughput Measurement of Morphology of Stored Red Blood Cells,â 2013) Performs segmentation and classification on RBCs in a microfluidic device â Sec. 3.4 âRonnebergerâ (âU-Net: Convolutional Networks for Biomedical Image Segmentation,â 2015) Teaches U-Net âSabourâ (âDynamic routing between capsules,â 2017) Teaches CapsNet Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examinerâs supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /GEOFFREY E SUMMERS/Examiner, Art Unit 2669