18243600. AUTOMATED ITEM INFORMATION ASSISTANCE FROM IMAGES (Maplebear Inc.)

AUTOMATED ITEM INFORMATION ASSISTANCE FROM IMAGES

Organization Name

Maplebear Inc.

Inventor(s)

Shubhanshu Mishra of Woodridge IL (US)

Gia Young of Livermore CA (US)

Jennie Morgan Burger of Tucson AZ (US)

Joseph Olivier of San Bruno CA (US)

Brent Luna of Oakland CA (US)

Yuanzheng Zhu of Smyrna GA (US)

Mackenzie Cala of San Francisco CA (US)

Armand Raquel-santos of San Rafael CA (US)

David Zandman of San Francisco CA (US)

Mia Martinez Barnett of Chicago IL (US)

Callie Bleckner of Oakland CA (US)

Mohammad Abdul-rahim of San Francisco CA (US)

AUTOMATED ITEM INFORMATION ASSISTANCE FROM IMAGES

This abstract first appeared for US patent application 18243600 titled 'AUTOMATED ITEM INFORMATION ASSISTANCE FROM IMAGES

Original Abstract Submitted

An online concierge system assists users in identifying additional information about items in an image. Image regions are identified in the image that may correspond to unknown items and an item search space is determined for detecting items in the image regions based on a context of the image, such as items in a warehouse or a list of items delivered to a customer. The identified items are used to retrieve relevant item information that is included in a prompt for a language model to extract relevant information for the item. As such, the process may automatically process the image into relevant textual information about the pictured items. Applications may be used to assist vision-impaired users in distinguishing delivered items or quickly identifying and evaluating relevant information about items.