dataengineering共152篇
5 Beginner-Friendly Projects to Learn Data Engineering (Using Free Tools)-拾光赋

5 Beginner-Friendly Projects to Learn Data Engineering (Using Free Tools)

5 Beginner-Friendly Projects to Learn Data Engineering (Using Free Tools),Title: 5 Beginner-Friendly Projects to Learn Data Engineering (Using Free Tools) CSV to SQL Migration Pipe...
kity的头像-拾光赋kity4天前
0487
️ Why Web Scraping in Python Is a Must-Have Skill in 2025-拾光赋

️ Why Web Scraping in Python Is a Must-Have Skill in 2025

️ Why Web Scraping in Python Is a Must-Have Skill in 2025 ,Want to grab real-world data at scale? Scrape prices, news, job posts, products — directly from the source. No API? No ...
kity的头像-拾光赋kity4天前
02810
Data Cleaning in Python, 2025 Edition-拾光赋

Data Cleaning in Python, 2025 Edition

Data Cleaning in Python, 2025 Edition,Messy datasets? Missing values? Duplicates? Inconsistent categories? 🧹 This guide shows you how to clean it all up using just pandas — and k...
kity的头像-拾光赋kity4天前
03010
How to treat secure data on lakehouse-拾光赋

How to treat secure data on lakehouse

How to treat secure data on lakehouse,In the modern data stack, the lakehouse has emerged as a hybrid solution that combines the scalability of a data lake with the transactional p...
kity的头像-拾光赋kity7天前
04314
Understanding Data Pipelines: The Backbone of Modern Data Systems-拾光赋

Understanding Data Pipelines: The Backbone of Modern Data Systems

Understanding Data Pipelines: The Backbone of Modern Data Systems, In today’s data-driven world, organizations are collecting vast amounts of data from various sources — websites...
kity的头像-拾光赋kity8天前
03110
Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel-拾光赋

Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel

Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel,Have you ever needed up-to-date job listings but struggled to find one clean source? In this proj...
kity的头像-拾光赋kity9天前
04513
Stock Data Extraction Using Apache Kafka-拾光赋

Stock Data Extraction Using Apache Kafka

Stock Data Extraction Using Apache Kafka,Overview This project utilizes Apache Kafka to extract stock data from the Polygon.io API and stores it in an Apache Cassandra database. It...
kity的头像-拾光赋kity9天前
0277
Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL-拾光赋

Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL

Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL,Introduction As part of mastering Apache Airflow, here is a documentation of a project where I extracted B...
kity的头像-拾光赋kity13天前
0365
Creating a new Airbyte connector from scratch-拾光赋

Creating a new Airbyte connector from scratch

Creating a new Airbyte connector from scratch,In this tutorial, we will create a new Airbyte connector from scratch using the Airbyte UI Connector Builder. You'll learn: How to ins...
kity的头像-拾光赋kity14天前
02214
Stop flattening your JSON just to export it to Excel-拾光赋

Stop flattening your JSON just to export it to Excel

Stop flattening your JSON just to export it to Excel,If you've ever needed to share structured Python data (like nested dicts or lists) with non-developers, chances are you've hit ...
kity的头像-拾光赋kity18天前
05113
Pandas Too Slow? Try These Fast Python Libraries for Data Analysis-拾光赋

Pandas Too Slow? Try These Fast Python Libraries for Data Analysis

Pandas Too Slow? Try These Fast Python Libraries for Data Analysis,Pandas is great — until it crashes with large data. Here’s what to use instead In my latest post, I explore mod...
kity的头像-拾光赋kity25天前
02112
Pandas Mindmap: A Visual Guide to DataFrame Manipulation-拾光赋

Pandas Mindmap: A Visual Guide to DataFrame Manipulation

Pandas Mindmap: A Visual Guide to DataFrame Manipulation,Pandas is an essential library for data manipulation and analysis in Python. This mindmap provides a structured visual appr...
kity的头像-拾光赋kity29天前
02610