Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sat Sep 18 12:14:17

	MIT to Host First Citizen Data Science Summit on September 20 \| Register For Free by /u/saik2363 1 hour ago.
	Best Programming Languages for Data Science by /u/techexpertsforall 5 hours ago.
	Cloud Native Data Pipelines using ArgoWorkflow by Robert Heise 7 hours ago.
	Standard/Best ways of moving data between Databases by /u/QuaternionHam 9 hours ago.
	What is the "modern data stack" at your company? by /u/knlph 12 hours ago.
	Data Quality: Anomaly Detections that Work with ThirdEye by /u/cyrilou242 13 hours ago.
	Interview assignment too big ? by /u/DrSnakee95 13 hours ago.
	I just had an interview with a Senior Data Engineer and the head of the Data Science department of a company and they said, in the next interview they'd see how I think. by /u/noNSFWcontent 14 hours ago.
	How UPS makes Data Driven Decisions for AI Innovation with Laura Patel, Advanced Analytics and Gregory Brown, VP R&D - Thoughts? by /u/DataGeek0 16 hours ago.
	CloudQuery: SQL for Cloud Infrastructure by /u/jekapats 16 hours ago.
	What are the innovations changing and shaping data engineering right now? by /u/Aromatic-Pay-3540 18 hours ago.
	How do you handle data versioning for reproducibility? by /u/britishbanana 19 hours ago.
	Help in selecting an offer from a German org from India for data engineer position by /u/International-Life17 19 hours ago.
	Datawarehouse Best Practice? by /u/Phantazein 21 hours ago.
	QuantumBlack Data Engineering? by /u/sa1amandra 21 hours ago.
	The Simple 5-Step Process for Creating a Winning Data Pipeline by /u/saik2363 22 hours ago.
	Suggestion: Thread for TED Talks by /u/stackedhats 22 hours ago.
	Being A Data Engineer: Expectations vs Reality by /u/nonkeymn 22 hours ago.
	rqlite: The Distributed Database Built on Raft and SQLite (Philip O'Toole) by CMU Database Group yesterday
	05 - Buffer Pools (CMU Databases Systems / Fall 2021) by CMU Database Group yesterday
	YAML Generator for Funnel YAML Files: Streamlining the Mobile Data Workflow Process by Jack Ning yesterday
	Deduplication of streaming data in Kafka ksqlDB or How to find distinct events in streaming… by Mikita Kandratsiuk 2 days ago.
	Anomaly Detection: Why Your Data Team Is Just Not That Into It by Barr Moses 2 days ago.
	04 - Database Storage II (CMU Databases Systems / Fall 2021) by CMU Database Group 3 days ago.
	How Postman Fixed a Missing Layer in Our Data Stack by Prudhvi Vasa 3 days ago.
	Setting The Stage For The Next Chapter Of The Cassandra Database - Episode 220 by tmacey 5 days ago.
	03 - Database Storage I (CMU Databases Systems / Fall 2021) by CMU Database Group 8 days ago.
	What is Data Ingestion? by Einav Baraban 9 days ago.
	A View From The Round Table Of Gartner's Cool Vendors - Episode 219 by tmacey 9 days ago.
	How to Track Metadata with Airflow Cluster Policies and Task Callbacks by Jonathan Barda 9 days ago.
	How technology is impacting data modelling by Maria Patrakova 11 days ago.
	Timelines by Vlad Rișcuția 11 days ago.
	Designing And Building Data Platforms As A Product - Episode 218 by tmacey 14 days ago.
	What is a Data Incident Commander? by Glen Willis 14 days ago.
	02 - Intermediate SQL (CMU Databases Systems / Fall 2021) by CMU Database Group 14 days ago.
	01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2021) by CMU Database Group 15 days ago.
	SingleStore: Separation of Storage and Compute for Transactions and Analytics (Joyo Victor) by CMU Database Group 15 days ago.
	An Enterprise Data Stack Using TypeDB by Daniel Crowe 15 days ago.
	Streaming Real-Time Analytics with Redis, AWS Fargate, and Dash Framework by Piyush Choudhary 15 days ago.
	Presto Powered Cloud Data Lakes At Speed Made Easy With Ahana - Episode 217 by tmacey 16 days ago.
	Enabling Seamless Kafka Async Queuing with Consumer Proxy by Yang Yang 17 days ago.
	Do Away With Data Integration Through A Dataware Architecture With Cinchy - Episode 216 by tmacey 21 days ago.
	How Data Shapes the Uber Rider App by Divya Babu Ravichandran 22 days ago.
	Decoupling Data Operations From Data Infrastructure Using Nexla - Episode 215 by tmacey 24 days ago.
	Building Scalable Streaming Pipelines for Near Real-Time Features by Feng Xu 24 days ago.
	Let Your Analysts Build A Data Lakehouse With Cuelake - Episode 214 by tmacey 28 days ago.
	Migrate And Modify Your Data Platform Confidently With Compilerworks - Episode 213 by tmacey 30 days ago.
	Prepare Your Unstructured Data For Machine Learning And Computer Vision Without The Toil Using Activeloop - Episode 212 by tmacey 34 days ago.
	Efficiently Managing the Supply and Demand on Uber’s Big Data Platform by Zheng Shao 35 days ago.
	Cost-Efficient Open Source Big Data Platform at Uber by Zheng Shao 37 days ago.
	Build Trust In Your Data By Understanding Where It Comes From And How It Is Used With Stemma - Episode 211 by tmacey 39 days ago.
	Challenges and Opportunities to Dramatically Reduce the Cost of Uber’s Big Data by Zheng Shao 39 days ago.
	How Uber Achieves Operational Excellence in the Data Quality Experience by Ying Zou 43 days ago.
	Containerizing Apache Hadoop Infrastructure at Uber by Mithun Mathew 57 days ago.
	Analyzing Customer Issues to Improve User Experience by Nimesh Agarwal 63 days ago.
	MonetDB: Scale Up Before You Scale Out (Martin Kersten) by CMU Database Group 75 days ago.
	Systems for Human Data Interaction (Eugene Wu) by CMU Database Group 86 days ago.
	PostgreSQL Optimizer Methodology (Robert Haas) by CMU Database Group 95 days ago.
	The Design of InfluxDB IOx: In-Memory Columnar Database Written in Rust with Apache Arrow (Paul Dix) by CMU Database Group 118 days ago.
	Noria: Fast Materialized Views for Fast Websites (Malte Schwarzkopf) by CMU Database Group 118 days ago.
	Under the Hood of an Exadata Transaction – How We Harnessed the Power of Persistent Memory (Jia Shi) by CMU Database Group 134 days ago.
	Automating Merchant Live Monitoring with Real-Time Analytics: Charon by Marco Vita 141 days ago.
	eXtremeDB: Deterministic Database Management in Mission-Critical Applications (Andrei Gorine) by CMU Database Group 142 days ago.
	LeanStore: In-Memory Data Management Beyond Main Memory (Viktor Leis) by CMU Database Group 150 days ago.
	My thoughts on the data mesh by Daniel Abadi 176 days ago.
	Uber’s Journey Toward Better Data Culture From First Principles by Krishna Puttaswamy 185 days ago.
	Turning Metadata Into Insights with Databook by Sunheng Taing 311 days ago.
	Operating Apache Pinot @ Uber Scale by Yupeng Fu 332 days ago.
	Inside Uber ATG’s Data Mining Operation: Identifying Real Road Scenarios at Scale for Machine Learning by Steffon Davis 472 days ago.
	Monitoring Data Quality at Scale with Statistical Modeling by Ye Henry Li 498 days ago.
	Building a Backtesting Service to Measure Model Performance at Uber-scale by Sam Xiao 582 days ago.
	Women in Data Science at Uber: Moving the World With Data in 2020—and Beyond by Emily Bailey 598 days ago.
	Designing a Production-Ready Kappa Architecture for Timely Data Stream Processing by Amey Chaugule 603 days ago.
	Engineering SQL Support on Apache Pinot at Uber by Haibo Wang 611 days ago.
	Uber Visualization Highlights: Displaying City Street Speed Clusters with SpeedsUp by Bryant Luong 624 days ago.
	It's time to rethink how we share data on the Web by Daniel Abadi 639 days ago.
	Uber’s Data Platform in 2019: Transforming Information to Intelligence by Nikhil Joshi 640 days ago.
	Productionizing Distributed XGBoost to Train Deep Tree Models with Large Data Sets at Uber by Joseph Wang 647 days ago.
	Introducing SLOG: Cheating the low-latency vs. strict serializability tradeoff by Daniel Abadi 652 days ago.
	The dangers of conditional consistency guarantees by Daniel Abadi 652 days ago.
	Introduction to Transaction Isolation Levels by Daniel Abadi 652 days ago.
	Correctness Anomalies Under Serializable Isolation by Daniel Abadi 652 days ago.
	Overview of Consistency Levels in Database Systems by Daniel Abadi 652 days ago.
	An explanation of the difference between Isolation levels vs. Consistency levels by Daniel Abadi 652 days ago.
	Partitioned consensus and its impact on Spanner’s latency by Daniel Abadi 653 days ago.
	It’s Time to Move on from Two Phase Commit by Daniel Abadi 653 days ago.
	NewSQL database systems are failing to guarantee consistency, and I blame Spanner by Daniel Abadi 653 days ago.
	Uber Visualization Highlights: How Urban Symphony Adds an Audio Dimension to Visualization by Jon Sadka 667 days ago.
	Taking City Visualization into the Third Dimension with Point Clouds, 3D Tiles, and deck.gl by Ib Green 689 days ago.
	Building a Better Big Data Architecture: Meet Uber’s Presto Team by Wayne Cunningham 709 days ago.
	Science at Uber: Making a Real-world Impact with Data Science by Molly Vorwerck 759 days ago.
	Less is More: Engineering Data Warehouse Efficiency with Minimalist Design by Ritesh Agrawal 765 days ago.
	Science at Uber: Powering Uber’s Ridesharing Technologies Through Mapping by Molly Vorwerck 773 days ago.
	Science at Uber: Bringing Research to the Roads by Molly Vorwerck 780 days ago.
	Science at Uber: Building a Data Science Platform at Uber by Wayne Cunningham 787 days ago.
	Making Apache Spark Effortless for All of Uber by Abhishek Modi 793 days ago.
	Visualizing City Cores with H3, Uber’s Open Source Geospatial Indexing System by Molly Vorwerck 802 days ago.
	Gaining Insights in a Simulated Marketplace with Machine Learning at Uber by Haoyang Chen 816 days ago.
	Using Causal Inference to Improve the Uber User Experience by Totte Harinen 821 days ago.
	Power On: Accelerating Uber’s Self-Driving Vehicle Development with Data by Steffon Davis 836 days ago.
	Second Uber Science Symposium: Exploring Advances in Behavioral Science by Laura Libby 851 days ago.
	Visualizing Traffic Safety with Uber Movement Data and Kepler.gl by Carsten Jacobsen 854 days ago.
	Improving Uber’s Mapping Accuracy with CatchME by Yuehai Xu 876 days ago.
	An analysis of the strengths and weaknesses of Apache Arrow by Daniel Abadi 1270 days ago.
	Apache Arrow vs. Parquet and ORC: Do we really need a third Apache project for columnar data representation? by Daniel Abadi 1417 days ago.
	Hazelcast and the Mythical PA/EC System by Daniel Abadi 1440 days ago.
	Scala Days 2017 - Jepsen Keynote by Aphyr 1598 days ago.
	Distributed consistency at scale: Spanner vs. Calvin by Daniel Abadi 1625 days ago.
	Jepsen: CockroachDB beta-20160829 by Aphyr 1674 days ago.
	Jepsen: MongoDB 3.4.0-rc3 by Aphyr 1683 days ago.
	IEEE Computer issue on the CAP Theorem by Daniel Abadi 1860 days ago.
	Jepsen: VoltDB 6.3 by Aphyr 1893 days ago.
	Jepsen: Crate 0.54.9 version divergence by Aphyr 1907 days ago.
	Jepsen: RethinkDB 2.2.3 reconfiguration by Aphyr 2065 days ago.
	Jepsen: RethinkDB 2.1.5 by Aphyr 2083 days ago.
	Why MongoDB, Cassandra, HBase, DynamoDB, and Riak will only let you perform transactions on a single data item by Daniel Abadi 2149 days ago.
	Jepsen: Percona XtraDB Cluster by Aphyr 2205 days ago.
	Jepsen: MariaDB Galera Cluster by Aphyr 2208 days ago.
	Jepsen: Chronos by Aphyr 2230 days ago.
	Jepsen: Aerospike by Aphyr 2328 days ago.
	Jepsen: Elasticsearch 1.5.0 by Aphyr 2335 days ago.
	Jepsen: MongoDB stale reads by Aphyr 2342 days ago.
	Jepsen: Elasticsearch by Aphyr 2651 days ago.
	Jepsen: etcd and Consul by Aphyr 2657 days ago.
	Jepsen: RabbitMQ by Aphyr 2661 days ago.
	Computational techniques in Knossos by Aphyr 2680 days ago.
	Strong consistency models by Aphyr 2683 days ago.
	Jepsen: Redis redux by Aphyr 2838 days ago.
	Jepsen: Strangeloop Hangout by Aphyr 2911 days ago.
	Jepsen: RICON East talk by Aphyr 2911 days ago.
	Jepsen: Cassandra by Aphyr 2915 days ago.
	Jepsen: Kafka by Aphyr 2916 days ago.
	Jepsen: NuoDB by Aphyr 2917 days ago.
	Jepsen: Zookeeper by Aphyr 2917 days ago.
	Automating Jepsen by Aphyr 3003 days ago.
	The network is reliable by Aphyr 3029 days ago.
	Asynchronous replication with failover by Aphyr 3041 days ago.
	Jepsen: final thoughts by Aphyr 3043 days ago.
	Jepsen: Riak by Aphyr 3043 days ago.
	Jepsen: MongoDB by Aphyr 3044 days ago.
	Jepsen: On the perils of network partitions by Aphyr 3044 days ago.
	Jepsen: Redis by Aphyr 3044 days ago.
	Jepsen: Postgres by Aphyr 3044 days ago.
	Is Upstart the right way to get college student start-ups funded? by Daniel Abadi 3236 days ago.
	Defending Matt Welsh’s 'Startup University' Post by Daniel Abadi 3370 days ago.
	If all these new DBMS technologies are so scalable, why are Oracle and DB2 still on top of TPC-C? A roadmap to end their dominance. by Daniel Abadi 3411 days ago.
	Replication and the latency-consistency tradeoff by Daniel Abadi 3571 days ago.
	Overview of the Oracle NoSQL Database by Daniel Abadi 3633 days ago.
	Hadoop's tremendous inefficiency on graph data management (and how to avoid it) by Daniel Abadi 3704 days ago.
	Why Sam Madden is wrong about peer review by Daniel Abadi 3774 days ago.
	Why I'm doing a start-up pre-tenure by Daniel Abadi 3825 days ago.

Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sat Sep 18 12:14:17