Patent Application 15752596 - INTERACTIVE 3D MAP WITH VIBRANT STREET VIEW

Title: INTERACTIVE 3D MAP WITH VIBRANT STREET VIEW
Application Information

Invention Title: INTERACTIVE 3D MAP WITH VIBRANT STREET VIEW
Application Number: 15752596
Submission Date: 2025-05-14T00:00:00.000Z
Effective Filing Date: 2018-02-14T00:00:00.000Z
Filing Date: 2018-02-14T00:00:00.000Z
National Class: 715
National Sub-Class: 720000
Examiner Employee Number: 91069
Art Unit: 2144
Tech Center: 2100
Rejection Summary

102 Rejections: 0
103 Rejections: 2
Cited Patents

The following patents were cited in the rejection:
US 0244648🔗
US 0106594🔗
Office Action Text



    DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
	
The action is in response to claims dated 4/9/2025
Claims pending in the case: 1, 4-6, 11-12, 23, 25-30
Cancelled claims: 2-3, 7-10, 13-22, 24


Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA  to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.  
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.



Claims 1, 4-6, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Thrun (US 2008/0106594), Vats (US 20140208272) and Kitatani (US 20140218288).

Regarding claim 1, Thrun teaches a method for providing interaction with a virtual object in a virtual space, the method comprising: 
- providing a panoramic video or a real-time video of the virtual space inside 3D graphics environment (Thrun: [7, 19]: virtual tour using panoramic images rendered in a 3-D space), wherein one or more portions of one or more frames of the panoramic video are clickable (Thrun: [21, 23, 43, 60, 63]: clickable panoramic video where the user may select a tag; [63]: tags “which are displayed simultaneously with the associated video frames”); 
- receiving a first user input over at least one of the portions of at least one of the frames of the panoramic video (Thrun: [21]: click on the panoramic video); and 
- generating and displaying a first view of a 3 dimensional model of the virtual object which is predefined for the particular portion of the one or more frames for which the first user input is received (Thrun: [9, 38, 60]: click on the panoramic video to display predefined content corresponding to the tag; [60]: “the display of a 3-D model in an interactive viewer”), 
However Thrun does not specifically recite, 
3D graphics environment; wherein displaying the 3 dimensional model in the 3D graphics environment;
Vats teaches,
wherein displaying the 3 dimensional model in the 3D graphics environment (Vats: Figs 18-19, [26-27, 45]: panoramic view of a place in 3D);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Thrun and Vats because the combination would enable using 3D graphics for displaying content in a 3D virtual environment that user can interact with. One of ordinary skill in the art would have been motivated to combine the teachings because the combination would give the user an improved experience by providing enhanced digital object viewing using 3D graphics and an interaction experience using an assistant as is possible in the physical environment (see Vats [1-2]);
Kitatani further teaches, providing inside 3D graphics environment; (Kitatani: Fig. 5, [2, 8, 96]: interactive map display in a virtual space where user may select a portion of the map to view associated content; [9]: a video may be displayed in the virtual space); It would have been obvious to one skilled in the art to use 3D graphics to generate a 3D space as taught in Vats. 
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Thrun, Vats and Kitatani because the combination would enable displaying a panoramic image/video and the user selected content in one unified virtual environment . One of ordinary skill in the art would have been motivated to combine the teachings because the combination would give the user a more immersed experience.

Regarding claim 4, Thrun, Vats and Kitatani teach the invention as claimed in claim 1 above, and further,
- displaying a virtual avatar of representative of the virtual space along with the panoramic video of the virtual space in the 3D graphics environment (Vats: Figs. 18-19, [7, 33, 45-46]: assistant in virtual environment); and 
- enabling conversation of an user with representative through video conferencing or through audio conferencing (Vats: [34, 44, 48-49]: conversation with assistant in virtual environment), Wherein the virtual avatar of the representative is shown when the audio conferencing is used for conversation, such that the virtual avatar is shown with facial and/or body expression (Vats: [45]: human like virtual assistant with facial expressions);

Regarding claim 5, Thrun, Vats and Kitatani teach the invention as claimed in claim 4 above, and further, wherein the virtual avatar is a 3 dimensional model which render in synchronization with input audio (Vats: [45-46]: avatar with synchronized lip movement and expressions).

Regarding claim 6, Thrun, Vats and Kitatani teach the invention as claimed in claim 4 above, and further, wherein the virtual avatar is a 2 dimensional image whose facial expression changes using image procession in synchronization with input audio of the representative (Vats: [45-46]: avatar with synchronized lip movement and expressions; [46]: “virtual assistant can also be an image or 3D model, where the virtual assistant (1901') is shown moving lips in response to a query.").

Regarding claim 29, Thrun, Vats and Kitatani teach the invention as claimed in claim 1 above, and further, 
- receiving a second user input, wherein the second user input are one or more interaction commands comprises interactions for understanding functionality of different parts of the 3D model (Vats: [32, 48]: user input commands); 
- identifying one or more interaction commands (Vats: [32-33, 48]; interact as in a real setup);
-in response to the identified one or more commands, rendering of corresponding interaction to 3D model of object using texture data, computer graphics data and selectively using sound data of the 3D- model of object (Vats: [32-33, 48]; interact as in a real setup) ; and 
- displaying the corresponding interaction to 3D model, wherein the interaction can emulate the interaction with mechanical and/or electronic and/or light emitting parts similar to real object (Vats: [32-33, 48]; interact as in a real setup).

Claims 11-12, 23, 25-28, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Thrun (US 2008/0106594), Vats (US 20140208272) and Kitatani (US 20140218288) in further view of Arfvidsson (US 2008/0244648).

Regarding claim 11, Thrun teaches, A method for providing an interactive … view comprising: 
- showing a panoramic video … inside a 3D graphics environment (Thrun: [7, 19]: virtual tour using panoramic images projected in a 3-D space), wherein one or more virtual premises shown in one or more frames of the panoramic video are clickable (Thrun: [21, 23, 43, 60, 63]: clickable panoramic video where the user may select a tag); 
- receiving a first user input over at least one of the virtual premises shown in at least one of the frames of the panoramic video (Thrun: [21]: click on the panoramic video); 
- loading a video or a panoramic image the virtual premises in …a 3D graphic environment for which the first user input is received (Thrun: [60, 63]: clickable panoramic video where the user may select a tag and the associated content is loaded; [60]: “the display of a 3-D model in an interactive viewer”); 
- providing a panoramic video or a real-time video of the virtual premises in the 3D graphics environment, wherein one or more portions of one or more frames of the panoramic video are clickable (Thrun: [21, 23, 43, 60, 63]: clickable panoramic video where the user may select a tag); 
- receiving a second user input over at least one of the portions of at least one of the frames of the panoramic video (Thrun: [21]: click on the panoramic video); and 
- generating and displaying a first view of a 3 dimensional model of the virtual object which is predefined for the particular portion of one or more frames for which the second user input is received (Thrun: [9, 38, 60]: click on the panoramic video to display predefined content corresponding to the tag; [60]: “the display of a 3-D model in an interactive viewer”), 
However Thrun does not specifically teach, video of street inside a 3D graphics environment;
wherein the 3 dimensional model of the virtual object is displayed in the 3D graphics environment;
Vats teaches,
wherein displaying the 3 dimensional model in the 3D graphics environment (Vats: Figs 18-19, [26-27]: panoramic view of a place in 3D);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Thrun and Vats because the combination would enable displaying content in a 3D virtual environment that user can interact with. One of ordinary skill in the art would have been motivated to combine the teachings because the combination would give the user an improved experience by providing enhanced digital object viewing and interaction experience using an assistant as is possible in the physical environment (see Vats [1-2]);
Kitatani further teaches, providing inside 3D graphics environment; (Kitatani: Fig. 5, [2, 8, 96]: interactive map display in a virtual space where user may select a portion of the map to view associated content; [9]: a video may be displayed in the virtual space); It would have been obvious to one skilled in the user to display a panoramic image/video of a world map in a virtual space for user interaction. 
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Thrun, Vats and Kitatani because the combination would enable displaying the panoramic image/video and the user selected content in one unified virtual environment . One of ordinary skill in the art would have been motivated to combine the teachings because the combination would give the user a more immersed experience;
Arfvidsson further teaches a street view and showing a panoramic video of a street (Arfvidsson: Figs. 2-7, [6, 20]: street views; Figs. 5-7 illustrate various premises labeled in the virtual space (e.g. Union Bank of California, Paul Hastings Tower, Los Angeles Central Library).
	It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Thrun, Vats and Kitatani and Arfvidsson because the combination would enable displaying panoramic view of street . One of ordinary skill in the art would have been motivated to combine the teachings because the combination would enable the user to update the displayed panoramic video in real time by interacting with a map, and vice versa (See Arfvidsson [25]).

Regarding claim 12, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 11 above, and further, 
- receiving user input for a geo location (Arfvidsson: [26]: Browser window 200 comprises a text-box 210 for a user to enter a street address); 
- loading a 2 dimensional or 3 dimensional map of a virtual space around the geo location (Arfvidsson: [25-27: street view; FIGS. 5-7 illustrate various premises labeled in the virtual space (e.g. Union Bank of California, Paul Hastings Tower, Los Angeles Central Library 26]: Browser window 200 comprises a text-box 210 for a user to enter a street address); 
- further showing the virtual space in the map representing the desired geographical location (Arfvidsson: [26]: display street); and 
- loading panoramic video of the street (Arfvidsson: [26]: displays as per view point). 

Regarding claim 23, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 11 above, and further, - displaying a virtual avatar of representative of the virtual space along with the panoramic video of the virtual space in the 3D graphics environment (Vats: Figs. 18-19, [7, 33, 45-46]: assistant in virtual environment); and 
- enabling conversation of an user with representative through video conferencing or through audio conferencing (Vats: [34, 44, 48-49]: conversation with assistant in virtual environment), wherein the virtual avatar of the representative is shown when the audio conferencing is used for conversation, such that the virtual avatar is shown with facial and/or body expression (Vats: [45]: human like virtual assistant with facial expressions);

Regarding claim 25, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 23 above, and further, wherein the virtual avatar is a 3 dimensional model which render in synchronization with input audio (Vats: [45-46]: avatar with synchronized lip movement and expressions).

Regarding claim 26, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 23 above, and further, wherein the virtual avatar is a 2 dimensional image whose facial expression changes using image procession in synchronization with input audio of the representative (Vats: [45-46]: avatar with synchronized lip movement and expressions; [46]: “virtual assistant can also be an image or 3D model, where the virtual assistant (1901') is shown moving lips in response to a query.").

Regarding claim 27, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 23 above, and further,  - loading a simulated representation of the user, generated from one or more photographs, or a 3D avatar of the user (Vats: [34]:  "user can upload his own photograph to generate a virtual simulation of himself," wherein the "simulated human 3D-model can walk . . . to a showroom or directly visit a product.").

Regarding claim 28, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 27 above, and further, wherein the simulated representation or the 3D avatar of the user is enabled to move within the street and/or the virtual premises and further enabled to interact with the virtual premises and/or other virtual avatars present in space (Vats: [33-35, 40-41]:  a simulated 3D model avatar walks and interacts with the objects as phone, computer, light, and steering a car).

Regarding claim 30, Thrun, Vats, Kitatani and Arfvidsson teach the invention as claimed in claim 11 above, and further, 
- receiving a third user input, wherein the third user input are one or more interaction commands comprises interactions for understanding functionality of different parts of the 3D model (Vats: [32, 48]: user interaction with a 3D object model, wherein “polygons along with associated texture of said 3D-model moves as per user command, and movement of 3D-model or its parts is achieved and displayed in real time … based on user input commands.”); 
- identifying one or more interaction commands (Vats: [32-33, 48]; interact as in a real setup); 
-in response to the one or more identified command/s, rendering of corresponding interaction to 3D model of object using texture data, computer graphics data and selectively using sound data of the 3D- model of object (Vats: [32-33, 48]; interact as in a real setup); and 
- displaying the corresponding interaction to 3D model wherein the interaction can emulate the interaction with mechanical and/or electronic and/or light emitting parts similar to real object (Vats: [32-33, 48]; interact as in a real setup).

Response to Arguments
Amendment overcomes the objection. The objection is respectfully withdrawn.
Applicants’ prior art arguments have been fully considered and are not persuasive. The applicant argues that “providing a panoramic video or a real-time video of the virtual space inside 3D graphics environment is different as the views taught in the art. The examiner finds that the limitations broadly claims providing a panoramic view inside or in a 3D environment.  Thrun teaches providing interactive panoramic image in a 3-D space by projecting it in 3D. Thus the teachings in Thrun reads on providing panoramic video in a 3-D environment. While Thrun does not specifically teach 3D graphics, it is known in the art that projection is done using 3D graphics. Vats teach this aspect i.e. projection by using 3D graphics. Kitatani further teaches user interaction and image frames being displayed inside 3D space.  
Thus the arts teach that providing panoramic image in 3D and in a 3D space. The cited arts also teach that the image may be interactive and that a 3D space may be generated by 3D graphics. Thus the combined teachings in the art read on the limitations as claimed. The limitations need to be more specific to overcome the rejection. 
The examiner adds that the specification merely mentions that 3D graphics may be used for rendering and specification paragraph [33] seems to indicate that the panoramic video view is in 3D. There is no details on the environment being separate from the displayed video let alone displayed inside an environment. Thus the examiner finds that the limitations as presented and interpreted based on the specification are obvious over the teachings in the prior arts.

Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure in attached 892.
THIS ACTION IS MADE FINAL.  Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).  
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.  In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action.  In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. 
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MANDRITA BRAHMACHARI whose telephone number is (571)272-9735. The examiner can normally be reached Monday to Friday, 11 am to 8 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached on 571 272 4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.





/Mandrita Brahmachari/Primary Examiner, Art Unit 2176