While database concepts like transactions, commits and rollbacks are necessary for traditional data warehousing workloads, they are not sufficient for modern data platforms and data-driven companies.
Project Nessie is a new open source metastore that builds on table formats such as Apache Iceberg and Delta Lake to deliver multi-table, multi-engine transactions. In addition, Nessie delivers Git-like semantics for data lakes, including commits, tags and branches. By using versioning concepts, users can work in an entirely new way, experimenting or preparing data without impacting the live view of the data, opening a whole world of possibilities for true DataOps on the data lake.
This talk will discuss the benefits of Nessie and Iceberg and how these technologies can work together in modern data platforms.
Ryan Murray (Dremio)
Ryan Murray is an open source engineer at Dremio in the office of the CTO. He previously served in the financial services industry doing everything from bond trader to data engineering lead. Ryan holds a PhD in theoretical physics and is an active open source contributor who dislikes it when data isnt accessible in an organisation. He is passionate about making customers successful and self-sufficient, and still one day dreams of winning the Stanley Cup