Explore
- #data-engineering 11
- #gcp 6
- #bigquery 5
- #devops 3
- #streaming 3
- #architecture 2
- #grafana 2
- #monitoring 2
- #eks 2
- #kubernetes 2
- #python 2
- #pubsub 2
- #kafka 2
- #dbt 2
- #airflow 2
- #sql 1
- #biglake 1
- #lakehouse 1
- #observability 1
- #opentelemetry 1
- #cicd 1
- #github-actions 1
- #aws 1
- #gke 1
- #fastapi 1
- #api 1
- #dataflow 1
- #analytics-engineering 1
- #data-modeling 1
- #dataproc 1
- #spark 1
- #elt 1
- #cloud-composer 1
- #orchestration 1
- #docker 1
- #cost-optimization 1
- #cloud-storage 1
- #bigtable 1
- #spanner 1
- #snowflake 1
- #postgresql 1
2026
Streaming Data into BigQuery with Pub/Sub and Dataflow — A Practical Guide
Build a real-time streaming pipeline from Pub/Sub to BigQuery using Dataflow, with windowing, error handling, and exactly-once semantics.
Kafka vs. Pub/Sub — Choosing a Streaming Backbone for Your Data Platform
A hands-on comparison of Apache Kafka and Google Pub/Sub covering throughput, ordering guarantees, ecosystem, and when to use each.
Real-Time Banking CDC Pipeline
Captures banking transaction changes in real-time using CDC, transforming operational data into analytics-ready models for business intelligence.