Just Enough Data | Newsletter 16
Just a normal evening in Portugal
Connector Builder by Airbyte
Airbyte users can now create ELT connectors without code in 10 minutes, using the Connector Builder. Address your team’s long tail of connector needs. It handles authentication, pagination, rate limiting, schema handling, transformation, response decoding...
Data Engineering Patterns: Write-Audit-Publish (WAP)
Write-Audit-Publish (WAP) is a pattern in data engineering that gives teams greater control over data quality. It was popularized by Netflix back in 2017 in a talk by Michelle Winters at the DataWorks Summit called “Whoops the Numbers are wrong! Scaling Data Quality @ Netflix.”
How do we Continuously Grow and Evolve in our roles
Event by Kamran Ali
Microsoft launches Fabric, a new end-to-end data and analytics platform
Microsoft today launched Microsoft Fabric, a new end-to-end data and analytics platform (not to be confused with Azure Service Fabric).
Databricks Releases Keynote Lineup and Generation AI Programming for 2023 Data + AI Summit
SAN FRANCISCO - June 1, 2023 – Databricks, the data and AI company, revealed the full agenda and lineup of featured speakers for the upcoming Data + AI Summit, a global event for the data community. On June 26-29, tens of thousands of data leaders, open source enthusiasts, and Databricks customers and partners will come together in person in San Francisco and virtually from around the world to learn about large language models (LLMs), the lakehouse paradigm, and Databricks’ latest product innovations and contributions to the open source community. Tickets are still available to join the data community in San Francisco; register here to attend the live event.
Test data quality during CI/CD development
Use this guide to install and set up Soda to test the quality of your data during your development lifecycle. Catch data quality issues in a GitHub pull request before merging data management changes, such as transformations, into production.
Mastering Monitoring
A Step-by-Step Guide to Using Prometheus and Grafana with CrateDB Cloud
In this interactive live demo, learn, step-by-step, how to use Prometheus and Grafana with CrateDB Cloud.
CrateDB Cloud users can implement their own monitoring. With the guidance of Walter Behmann, Engineer at CrateDB Cloud, you will learn how to monitor a cloud service, specifically CrateDB, using Prometheus and Grafana.
By the end of this webinar, you will be equipped to level up your monitoring game.
Welcome Okera: Adopting an AI-centric approach to governance
Okera, the world's first AI-centric data governance platform. Okera solves data privacy and governance challenges across the spectrum of data and AI
Thought for the day
“I am comfortable here” has killed more dreams of software engineer than anything else.