Pipeline To Insights

Pipeline To Insights

Home
Notes
Chat
Data Engineering Lifecycle
SQL Optimisation
DE Interview Preperation
Data Quality
Data Career Transition
data build tool (dbt) series
DevOps for Data Engineers
Archive
Leaderboard
About
Most Popular
View all
Kubernetes for Data Engineers
For many data engineers, Kubernetes feels like a mysterious DevOps black box, something we know exists and hear about all the time, but don’t fully…
 • 
Erfan Hesami
Apache Spark Fundamentals for Data Engineers
A beginner-friendly guide to understanding Spark’s architecture, ecosystem, and step-by-step execution using a practical example.
 • 
Erfan Hesami
Getting Started with OpenMetadata: An Open-Source Data Catalogue Solution
What is OpenMetadata, how to deploy it, our experience with it, and many more.
 • 
Erfan Hesami
A Data Engineer’s Guide to Vector Databases (Part 2): Full-Text, Semantic, Hybrid Search and Reranking with pgvector
Getting Started with PGVector: Bringing Vector Search to PostgreSQL
 • 
Erfan Hesami
A Data Engineer’s Guide to Vector Databases (Part 1): Core Concepts Before Building AI-Powered Applications
A practical foundation for data engineers transitioning from relational to vector-based data systems.
 • 
Erfan Hesami
Storage Fundamentals for Data Engineers
From the storage hierarchy to cloud storage types, what every data engineer needs to know to choose the right solution and interact with it.
 • 
Erfan Hesami
Pandas vs. Polars vs. DuckDB vs. PySpark: Benchmarking Libraries with Real Experiments
A hands-on benchmark comparing CSV loading, filtering, aggregation, sorting, and joins on 5GB (100M rows).
 • 
Erfan Hesami
Pipeline To Insights
Pipeline To Insights
Pipeline To Insights is a community-driven blog by passionate Data Engineers, sharing real-world experiences, technical tutorials, and personal reflections to inspire growth and continuous learning in the evolving world of data and AI.
Recommendations
View all 17
SeattleDataGuy’s Newsletter
SeattleDataGuy’s Newsletter
SeattleDataGuy
Data Gibberish
Data Gibberish
Yordan Ivanov
VuTrinh.
VuTrinh.
Vu Trinh
ShortlistHR
ShortlistHR
Kerry Peters
Joe Reis
Joe Reis
Joe Reis

Pipeline To Insights

AboutArchiveRecommendationsSitemap
© 2025 Erfan Hesami · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture