Computer Science > Databases
[Submitted on 5 Jul 2021 (v1), last revised 7 Oct 2022 (this version, v7)]
Title:PandaDB: Understanding Unstructured Data in Graph Database
View PDFAbstract:Unstructured data(e.g., images, videos, PDF files, etc.) contain semantic information, for example, the facial feature of a person and the plate number of a vehicle. There could be semantic relationships between data items which are not explicitly represented. For example, a person's face may appear in two irrelevant photos. Also, much information is represented as structured data(e.g., the person's name and age). End-users prefer to query the semantic information from unstructured data together with structured data based on the potential relationships among them. However, due to the lack of a unified database system for structured and unstructured data, developers have to comprise multiple systems and runtime together to answer these queries. In this work, we build an open-source graph database named PandaDB to consistently manage and query structured and unstructured data. We first introduce a graph data model to manage structured and unstructured data, then propose a new query language to understand the semantics of the unstructured data in the graph. Next, we develop a new cost model and related query optimization techniques to speed up the unstructured data processing pipeline. Finally, we optimize the unstructured data storage and provide the index to speed up the query processing over unstructured data. PandaDB is widely used in industrial applications like FinTech, Knowledge Graph, and Recommendation System. The results show PandaDB can support a large scale of unstructured data query processing in a graph.
Submission history
From: Zihao Zhao [view email][v1] Mon, 5 Jul 2021 12:13:38 UTC (3,419 KB)
[v2] Thu, 16 Dec 2021 10:14:20 UTC (8,804 KB)
[v3] Fri, 17 Dec 2021 04:19:43 UTC (8,811 KB)
[v4] Fri, 25 Feb 2022 13:16:00 UTC (9,007 KB)
[v5] Mon, 28 Feb 2022 11:17:44 UTC (18,237 KB)
[v6] Wed, 6 Jul 2022 13:10:21 UTC (9,235 KB)
[v7] Fri, 7 Oct 2022 23:31:30 UTC (2,454 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.