Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sun Oct 17 08:11:36

	Until 6th of November Google Cloud Professional Certs, Specializations and standalone courses are FREE by /u/awsconsultant 1 hour ago.
	Are microsoft data engg cert any good for ppl with no experience? by /u/Worried-Cobbler-3858 6 hours ago.
	Part 2 - Preparing for the Azure Data Engineer Certificate DP203 - Notes for Azure Data Lakes Gen2 by /u/noNSFWcontent 6 hours ago.
	Offered DE Job... Feeling under-qualified by /u/sillythebunny 7 hours ago.
	All my WFH people - what internet provider do you use and what plan ? by /u/The_Alpacas 7 hours ago.
	Customer facing services by /u/CapitalistZ 8 hours ago.
	SQL Puzzles by /u/jrz1977 10 hours ago.
	Table vs View materialization by /u/nobel-001 13 hours ago.
	Part 1 - Preparing for the Azure Data Engineer Certificate DP203 - Notes for Setting up SQL Server and Storage Account. by /u/noNSFWcontent 13 hours ago.
	Has anyone done the Data Engineering Track on Data Camp - the testing module.... X-( by /u/strahan47 13 hours ago.
	Day In The Life Of A Data Engineer - What Do Data Engineers Do? by /u/SeattleDataGuy 15 hours ago.
	Hacker Rank et al. by /u/welschii 16 hours ago.
	Is using pandas considered terrible practice in ELT? by /u/UnderstandingFit9152 17 hours ago.
	Are there any study guides for system design interviews for data engineers? by /u/statistical_engineer 18 hours ago.
	Right way to leetcode for Data Engineering by /u/Southern-Door-1104 18 hours ago.
	How we use events and Kafka to make blockchain more accessible by /u/farfaraway 19 hours ago.
	Hey DE Fam, need your help here. Tell me know what do you think in comments below. by /u/Complete_Solution_83 20 hours ago.
	Stream Processing and Apache Flink news at Flink Forward Global 2021 by /u/Marksfik 21 hours ago.
	Data Engineering for a Storage guy. by /u/netizentrotter 22 hours ago.
	My Experience as a Data Engineer vs. a Data Scientist by Vicky Yu yesterday
	Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data - Episode 230 by tmacey yesterday
	MySQL vs Cassandra DB by Mike Wolfe 2 days ago.
	Simulating Continuous Learning Models with Airflow by Hari Devanathan 2 days ago.
	Back Up your Valuable BigQuery Views and Scheduled Queries by Krisjan Oldekamp 3 days ago.
	How And Why To Become Data Driven As A Business - Episode 229 by tmacey 3 days ago.
	Improving HDFS I/O Utilization for Efficiency by Kevin Cheng 3 days ago.
	How to Count Things with dbt - Data Build Tool (Drew Banin) by CMU Database Group 4 days ago.
	12 - Query Execution II (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 4 days ago.
	Collaboration Between Data Scientists and Data Engineers Made Simple by Yaron Vazana 4 days ago.
	The Most for Your Money from Amazon S3 by Vel Lesikov 4 days ago.
	Easy data mapping using AirMap by Erik Yan 5 days ago.
	Aggregations on time-series data with Pandas by Oleg Żero 7 days ago.
	Make Your Business Metrics Reusable With Open Source Headless BI Using Metriql - Episode 228 by tmacey 8 days ago.
	Not All Vector Databases Are Made Equal by Dmitry Kan 9 days ago.
	3 Steps for a Successful Data Migration by Mark Grover 9 days ago.
	11 - Query Execution I (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 10 days ago.
	BodoSQL: Fully Integrated Python and SQL with HPC Performance and Scalability (Ehsan Totoni) by CMU Database Group 11 days ago.
	Adding Support For Distributed Transactions To The Redpanda Streaming Engine - Episode 227 by tmacey 11 days ago.
	The TileDB Universal Database (Stavros Papadopoulos) by CMU Database Group 11 days ago.
	10 - Join Algorithms (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 12 days ago.
	09 - Sorting & Aggregations (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 14 days ago.
	Building Real-Time Data Platforms For Large Volumes Of Information With Aerospike - Episode 226 by tmacey 14 days ago.
	08 - Index Concurrency Control (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 14 days ago.
	Delivering Your Personal Data Cloud With Prifina - Episode 225 by tmacey 17 days ago.
	Digging Into Data Reliability Engineering - Episode 224 by tmacey 21 days ago.
	Massively Parallel Data Processing In Python Without The Effort Using Bodo - Episode 223 by tmacey 22 days ago.
	07 - Tree Indexes (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 22 days ago.
	Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot by Jacob Tsafatinos 23 days ago.
	Google Napa: Scalable Data Warehousing with Robust Query Performance (Jagan Sankaranarayanan) by CMU Database Group 23 days ago.
	06 - Hash Tables (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 24 days ago.
	03 - Database Storage I (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 25 days ago.
	04 - Database Storage II (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 25 days ago.
	05 - Buffer Pools (CMU Intro to Database Systems / Fall 2021) by CMU Database Group 25 days ago.
	Declarative Machine Learning Without The Operational Overhead Using Continual - Episode 222 by tmacey 27 days ago.
	An Exploration Of The Data Engineering Requirements For Bioinformatics - Episode 221 by tmacey 27 days ago.
	rqlite: The Distributed Database Built on Raft and SQLite (Philip O'Toole) by CMU Database Group 30 days ago.
	YAML Generator for Funnel YAML Files: Streamlining the Mobile Data Workflow Process by Jack Ning 30 days ago.
	Streaming Real-Time Analytics with Redis, AWS Fargate, and Dash Framework by Piyush Choudhary 44 days ago.
	Enabling Seamless Kafka Async Queuing with Consumer Proxy by Yang Yang 46 days ago.
	How Data Shapes the Uber Rider App by Divya Babu Ravichandran 51 days ago.
	Building Scalable Streaming Pipelines for Near Real-Time Features by Feng Xu 53 days ago.
	Efficiently Managing the Supply and Demand on Uber’s Big Data Platform by Zheng Shao 64 days ago.
	Cost-Efficient Open Source Big Data Platform at Uber by Zheng Shao 66 days ago.
	Challenges and Opportunities to Dramatically Reduce the Cost of Uber’s Big Data by Zheng Shao 68 days ago.
	How Uber Achieves Operational Excellence in the Data Quality Experience by Ying Zou 72 days ago.
	Containerizing Apache Hadoop Infrastructure at Uber by Mithun Mathew 86 days ago.
	Analyzing Customer Issues to Improve User Experience by Nimesh Agarwal 92 days ago.
	Automating Merchant Live Monitoring with Real-Time Analytics: Charon by Marco Vita 170 days ago.
	My thoughts on the data mesh by Daniel Abadi 205 days ago.
	Uber’s Journey Toward Better Data Culture From First Principles by Krishna Puttaswamy 214 days ago.
	Turning Metadata Into Insights with Databook by Sunheng Taing 340 days ago.
	Operating Apache Pinot @ Uber Scale by Yupeng Fu 361 days ago.
	Inside Uber ATG’s Data Mining Operation: Identifying Real Road Scenarios at Scale for Machine Learning by Steffon Davis 501 days ago.
	Monitoring Data Quality at Scale with Statistical Modeling by Ye Henry Li 527 days ago.
	Building a Backtesting Service to Measure Model Performance at Uber-scale by Sam Xiao 611 days ago.
	Women in Data Science at Uber: Moving the World With Data in 2020—and Beyond by Emily Bailey 627 days ago.
	Designing a Production-Ready Kappa Architecture for Timely Data Stream Processing by Amey Chaugule 632 days ago.
	Engineering SQL Support on Apache Pinot at Uber by Haibo Wang 640 days ago.
	Uber Visualization Highlights: Displaying City Street Speed Clusters with SpeedsUp by Bryant Luong 653 days ago.
	It's time to rethink how we share data on the Web by Daniel Abadi 668 days ago.
	Uber’s Data Platform in 2019: Transforming Information to Intelligence by Nikhil Joshi 669 days ago.
	Productionizing Distributed XGBoost to Train Deep Tree Models with Large Data Sets at Uber by Joseph Wang 676 days ago.
	Introducing SLOG: Cheating the low-latency vs. strict serializability tradeoff by Daniel Abadi 681 days ago.
	The dangers of conditional consistency guarantees by Daniel Abadi 681 days ago.
	Introduction to Transaction Isolation Levels by Daniel Abadi 681 days ago.
	Correctness Anomalies Under Serializable Isolation by Daniel Abadi 681 days ago.
	Overview of Consistency Levels in Database Systems by Daniel Abadi 681 days ago.
	An explanation of the difference between Isolation levels vs. Consistency levels by Daniel Abadi 681 days ago.
	Partitioned consensus and its impact on Spanner’s latency by Daniel Abadi 682 days ago.
	It’s Time to Move on from Two Phase Commit by Daniel Abadi 682 days ago.
	NewSQL database systems are failing to guarantee consistency, and I blame Spanner by Daniel Abadi 682 days ago.
	Uber Visualization Highlights: How Urban Symphony Adds an Audio Dimension to Visualization by Jon Sadka 696 days ago.
	Taking City Visualization into the Third Dimension with Point Clouds, 3D Tiles, and deck.gl by Ib Green 718 days ago.
	Building a Better Big Data Architecture: Meet Uber’s Presto Team by Wayne Cunningham 738 days ago.
	Science at Uber: Making a Real-world Impact with Data Science by Molly Vorwerck 788 days ago.
	Less is More: Engineering Data Warehouse Efficiency with Minimalist Design by Ritesh Agrawal 794 days ago.
	Science at Uber: Powering Uber’s Ridesharing Technologies Through Mapping by Molly Vorwerck 802 days ago.
	Science at Uber: Bringing Research to the Roads by Molly Vorwerck 809 days ago.
	Science at Uber: Building a Data Science Platform at Uber by Wayne Cunningham 816 days ago.
	Making Apache Spark Effortless for All of Uber by Abhishek Modi 822 days ago.
	Visualizing City Cores with H3, Uber’s Open Source Geospatial Indexing System by Molly Vorwerck 831 days ago.
	Gaining Insights in a Simulated Marketplace with Machine Learning at Uber by Haoyang Chen 845 days ago.
	Using Causal Inference to Improve the Uber User Experience by Totte Harinen 850 days ago.
	Power On: Accelerating Uber’s Self-Driving Vehicle Development with Data by Steffon Davis 865 days ago.
	Second Uber Science Symposium: Exploring Advances in Behavioral Science by Laura Libby 880 days ago.
	An analysis of the strengths and weaknesses of Apache Arrow by Daniel Abadi 1299 days ago.
	Apache Arrow vs. Parquet and ORC: Do we really need a third Apache project for columnar data representation? by Daniel Abadi 1446 days ago.
	Hazelcast and the Mythical PA/EC System by Daniel Abadi 1469 days ago.
	Scala Days 2017 - Jepsen Keynote by Aphyr 1627 days ago.
	Distributed consistency at scale: Spanner vs. Calvin by Daniel Abadi 1654 days ago.
	Jepsen: CockroachDB beta-20160829 by Aphyr 1703 days ago.
	Jepsen: MongoDB 3.4.0-rc3 by Aphyr 1712 days ago.
	IEEE Computer issue on the CAP Theorem by Daniel Abadi 1889 days ago.
	Jepsen: VoltDB 6.3 by Aphyr 1922 days ago.
	Jepsen: Crate 0.54.9 version divergence by Aphyr 1936 days ago.
	Jepsen: RethinkDB 2.2.3 reconfiguration by Aphyr 2094 days ago.
	Jepsen: RethinkDB 2.1.5 by Aphyr 2112 days ago.
	Why MongoDB, Cassandra, HBase, DynamoDB, and Riak will only let you perform transactions on a single data item by Daniel Abadi 2178 days ago.
	Jepsen: Percona XtraDB Cluster by Aphyr 2234 days ago.
	Jepsen: MariaDB Galera Cluster by Aphyr 2237 days ago.
	Jepsen: Chronos by Aphyr 2259 days ago.
	Jepsen: Aerospike by Aphyr 2357 days ago.
	Jepsen: Elasticsearch 1.5.0 by Aphyr 2364 days ago.
	Jepsen: MongoDB stale reads by Aphyr 2371 days ago.
	Jepsen: Elasticsearch by Aphyr 2680 days ago.
	Jepsen: etcd and Consul by Aphyr 2686 days ago.
	Jepsen: RabbitMQ by Aphyr 2690 days ago.
	Computational techniques in Knossos by Aphyr 2709 days ago.
	Strong consistency models by Aphyr 2712 days ago.
	Jepsen: Redis redux by Aphyr 2867 days ago.
	Jepsen: Strangeloop Hangout by Aphyr 2940 days ago.
	Jepsen: RICON East talk by Aphyr 2940 days ago.
	Jepsen: Cassandra by Aphyr 2944 days ago.
	Jepsen: Kafka by Aphyr 2945 days ago.
	Jepsen: NuoDB by Aphyr 2946 days ago.
	Jepsen: Zookeeper by Aphyr 2946 days ago.
	Automating Jepsen by Aphyr 3032 days ago.
	The network is reliable by Aphyr 3058 days ago.
	Asynchronous replication with failover by Aphyr 3070 days ago.
	Jepsen: final thoughts by Aphyr 3072 days ago.
	Jepsen: Riak by Aphyr 3072 days ago.
	Jepsen: MongoDB by Aphyr 3073 days ago.
	Jepsen: On the perils of network partitions by Aphyr 3073 days ago.
	Jepsen: Redis by Aphyr 3073 days ago.
	Jepsen: Postgres by Aphyr 3073 days ago.
	Is Upstart the right way to get college student start-ups funded? by Daniel Abadi 3265 days ago.
	Defending Matt Welsh’s 'Startup University' Post by Daniel Abadi 3399 days ago.
	If all these new DBMS technologies are so scalable, why are Oracle and DB2 still on top of TPC-C? A roadmap to end their dominance. by Daniel Abadi 3440 days ago.
	Replication and the latency-consistency tradeoff by Daniel Abadi 3600 days ago.
	Overview of the Oracle NoSQL Database by Daniel Abadi 3662 days ago.
	Hadoop's tremendous inefficiency on graph data management (and how to avoid it) by Daniel Abadi 3733 days ago.
	Why Sam Madden is wrong about peer review by Daniel Abadi 3803 days ago.
	Why I'm doing a start-up pre-tenure by Daniel Abadi 3854 days ago.

Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sun Oct 17 08:11:36