View source for 18456675. LOW LATENCY INGESTION INTO A DATA SYSTEM simplified abstract (Snowflake Inc.)

=LOW LATENCY INGESTION INTO A DATA SYSTEM=


==Organization Name==

[[:Category:Snowflake Inc.|Snowflake Inc.]]


[[Category:Snowflake Inc.]]

==Inventor(s)==

[[:Category:Dhiraj Gupta of San Mateo CA (US)|Dhiraj Gupta of San Mateo CA (US)]][[Category:Dhiraj Gupta of San Mateo CA (US)]]

[[:Category:Subramanian Muralidhar of Mercer Island WA (US)|Subramanian Muralidhar of Mercer Island WA (US)]][[Category:Subramanian Muralidhar of Mercer Island WA (US)]]

==LOW LATENCY INGESTION INTO A DATA SYSTEM - A simplified explanation of the abstract==


This abstract first appeared for US patent application 18456675 titled 'LOW LATENCY INGESTION INTO A DATA SYSTEM

==Simplified Explanation==

Described in this patent application are techniques for enhancing the transfer of metadata from a metadata database to a data system's database, such as a data warehouse. The metadata is stored in the metadata database with a version stamp and a partition identifier. Multiple readers can scan the metadata database based on these values to export the metadata to a cloud storage location. The exported data can then be automatically ingested into the database, which includes a journal and snapshot table.

* Techniques for improving transfer of metadata from a metadata database to a data system's database
* Metadata is stored in the metadata database with a version stamp and a partition identifier
* Multiple readers can scan the metadata database based on version stamp and partition identifier values
* Exported metadata is stored in a cloud storage location
* Exported data is automatically ingested into the database, which includes a journal and snapshot table

== Potential Applications ==
* Data warehousing
* Database management systems
* Cloud storage and data transfer

== Problems Solved ==
* Efficient transfer of metadata from a metadata database to a data system's database
* Simplified export and ingestion process
* Improved organization and management of metadata

== Benefits ==
* Streamlined transfer process reduces manual effort and potential errors
* Faster and more efficient data transfer and ingestion
* Improved organization and accessibility of metadata


==Original Abstract Submitted==

Described herein are techniques for improving transfer of metadata from a metadata database to a database stored in a data system, such as a data warehouse. The metadata may be written into the metadata database with a version stamp, which is monotonic increasing register value, and a partition identifier, which can be generated using attribute values of the metadata. A plurality of readers can scan the metadata database based on version stamp and partition identifier values to export the metadata to a cloud storage location. From the cloud storage location, the exported data can be auto ingested into the database, which includes a journal and snapshot table.



[[Category:G06F16/27]]
[[Category:G06F16/18]]
[[Category:G06F16/2453]]
[[Category:G06F16/25]]