Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sat Jun 19 12:13:02

	What is best book for SQL? by /u/SubstantialDurian5 17 minutes ago.
	Advice regarding data engineer jobs by /u/segment_fault930 5 hours ago.
	Can anyone review my Github projects and resume? I am a recent masters graduate and looking to get into data engineering. Any help/guidance would be great. by /u/tanaybh0510 7 hours ago.
	is windows enough for big data(Excluding wsl) by /u/Shyprogrammer1 8 hours ago.
	survey for developers / engineers facing problems around documentation / knowledge transfer by /u/guru223 9 hours ago.
	I will soon be interviewed by a Project Manager to be a Fresh Grad EDW Engineer , What questions should I expect ? by /u/Redxer 10 hours ago.
	How does your team deploy models? by /u/Beertarian 18 hours ago.
	Help with SQL downsampling by /u/Weekly_Anteater1941 20 hours ago.
	Is there an explicit definition for "Data Infrastructure" work? by /u/dolphinday2 20 hours ago.
	Airlow or paid GUI ETL tool? by /u/Craftyzebra1992 20 hours ago.
	Bring Order To The Chaos Of Your Unstructured Data Assets With Unstruk - Episode 196 by tmacey yesterday
	Handling Flaky Unit Tests in Java by Ravi Agarwal 3 days ago.
	PostgreSQL Optimizer Methodology (Robert Haas) by CMU Database Group 4 days ago.
	Accelerating Machine Learning Training And Delivery With In-Database ML - Episode 195 by tmacey 4 days ago.
	Taking A Tour Of The Google Cloud Platform For Data And Analytics - Episode 194 by tmacey 7 days ago.
	The Evolution of Data Science Workbench by Peng Du 8 days ago.
	Make Sure Your Records Are Reliable With The BookKeeper Distributed Storage Layer - Episode 193 by tmacey 10 days ago.
	Scaling of Uber’s API gateway by Abhishek Parwal 10 days ago.
	Build Your Analytics With A Collaborative And Expressive SQL IDE Using Querybook - Episode 192 by tmacey 15 days ago.
	Making Data Pipelines Self-Serve For Everyone With Shipyard - Episode 191 by tmacey 17 days ago.
	MonetDB: Scale Up Before You Scale Out (Martin Kersten) by CMU Database Group 18 days ago.
	Paving The Road For Fast Analytics On Distributed Clouds With The Yellowbrick Data Warehouse - Episode 190 by tmacey 22 days ago.
	Easily Build Advanced Similarity Search With The Pinecone Vector Database - Episode 189 by tmacey 24 days ago.
	The Design of InfluxDB IOx: In-Memory Columnar Database Written in Rust with Apache Arrow (Paul Dix) by CMU Database Group 27 days ago.
	Noria: Fast Materialized Views for Fast Websites (Malte Schwarzkopf) by CMU Database Group 27 days ago.
	Fraud Detection: Using Relational Graph Learning to Detect Collusion by Xinyu Hu 28 days ago.
	A Holistic Approach To Data Governance Through Self Reflection At Collibra - Episode 188 by tmacey 29 days ago.
	The Architecture of Uber’s API gateway by Madan Thangavelu 30 days ago.
	Unlocking The Power of Data Lineage In Your Platform with OpenLineage - Episode 187 by tmacey 31 days ago.
	Introducing Orbit, An Open Source Package for Time Series Inference and Forecasting by Edwin Ng 35 days ago.
	pprof++: A Go Profiler with Hardware Performance Monitoring by Milind Chabbi 38 days ago.
	Under the Hood of an Exadata Transaction – How We Harnessed the Power of Persistent Memory (Jia Shi) by CMU Database Group 43 days ago.
	Optimal Feature Discovery: Better, Leaner Machine Learning Models Through Information Theory by Adam Wang 43 days ago.
	SingleStore: Separation of Storage and Compute for Transactions and Analytics (Joyo Victor) by CMU Database Group 48 days ago.
	Automating Merchant Live Monitoring with Real-Time Analytics: Charon by Marco Vita 50 days ago.
	eXtremeDB: Deterministic Database Management in Mission-Critical Applications (Andrei Gorine) by CMU Database Group 51 days ago.
	Freight Pricing with a Controlled Markov Decision Process by Guillaume De Roo 52 days ago.
	LeanStore: In-Memory Data Management Beyond Main Memory (Viktor Leis) by CMU Database Group 59 days ago.
	Query Processing in Google BigQuery (Hossein Ahmadi + Aleksandras Surna) by CMU Database Group 60 days ago.
	FASTER: Efficient State Management for the Modern Edge-Cloud (Badrish Chandramouli) by CMU Database Group 63 days ago.
	Flipr: Making Changes Quickly and Safely at Scale by Andy Maule 67 days ago.
	NoisePage: The Self-Driving Database Management System (Lin Ma) by CMU Database Group 85 days ago.
	My thoughts on the data mesh by Daniel Abadi 85 days ago.
	HarperDB’s Data Storage Journey: From File System to LMDB (Kyle Bernhardy) by CMU Database Group 90 days ago.
	Celebration of Mike Stonebraker #8 - Students (Magda Balazinska, Andy Pavlo) by CMU Database Group 93 days ago.
	Celebration of Mike Stonebraker #10 - Closing Remarks (Mike Stonebraker) by CMU Database Group 93 days ago.
	Uber’s Journey Toward Better Data Culture From First Principles by Krishna Puttaswamy 94 days ago.
	Celebration of Mike Stonebraker #9 - Open Mic + Messages by CMU Database Group 97 days ago.
	Navigating to the Technical Program Management and Learning Team 98 days ago.
	Elastic Deep Learning with Horovod on Ray by Travis Addair 102 days ago.
	Applying Machine Learning in Internal Audit with Sparsely Labeled Data by Jesse He 108 days ago.
	How Uber Deals with Large iOS App Size by Milind Chabbi 112 days ago.
	Evolving Schemaless into a Distributed SQL Database by Ovais Tariq 115 days ago.
	Fast and Reliable Schema-Agnostic Log Analytics Platform by Chao Wang 119 days ago.
	Uber’s Real-time Data Intelligence Platform At Scale: Improving Gairos Scalability/Reliability by Gang Zhao 150 days ago.
	The Journey Towards Metric Standardization by Xiaodong Wang 157 days ago.
	Disaster Recovery for Multi-Region Kafka at Uber by Yupeng Fu 179 days ago.
	Uber’s Real-Time Push Platform by Uday Kiran Medisetty 182 days ago.
	No Code Workflow Orchestrator for Building Batch & Streaming Pipelines at Scale by Sriharsha Chintalapani 189 days ago.
	Horovod v0.21: Optimizing Network Utilization with Local Gradient Aggregation and Grouped Allreduce by Travis Addair 206 days ago.
	Turning Metadata Into Insights with Databook by Sunheng Taing 220 days ago.
	Meet the 2020 Safety Engineering Interns: COVID Edition by Safety Engineering Interns 232 days ago.
	Operating Apache Pinot @ Uber Scale by Yupeng Fu 241 days ago.
	Building from the Baltics: Meet the Uber Engineering Team in Vilnius, Lithuania by Ieva Sapokaite 247 days ago.
	Ludwig v0.3 Introduces Hyperparameter Optimization, Transformers and TensorFlow 2 support by Travis Addair 255 days ago.
	Revolutionizing Money Movements at Scale with Strong Data Consistency by Aakriti Singla 259 days ago.
	Spearheading Open Source: A Conversation with Jim Jagielski, Staff Technical Program Manager with the Uber Open Source Program Office by Uber Open Source Office 267 days ago.
	Designing Edge Gateway, Uber’s API Lifecycle Management Platform by Madan Thangavelu 304 days ago.
	Standing for Safety: Meet the Uber Sao Paulo Tech Team by Camila Carvalho 325 days ago.
	Introducing Domain-Oriented Microservice Architecture by Adam Gluck 330 days ago.
	Engineering Failover Handling in Uber’s Mobile Networking Infrastructure by Sivabalan Narayanan 339 days ago.
	Fiber: Distributed Computing for AI Made Simple by Jiale Zhi 353 days ago.
	Editing Massive Geospatial Data Sets with nebula.gl by Georgios Karnas 360 days ago.
	Profiles in Coding: Diana Yanakiev, Uber ATG, Pittsburgh by Bea Schuster 367 days ago.
	Building a Large-scale Transactional Data Lake at Uber Using Apache Hudi by Nishith Agarwal 374 days ago.
	Introducing Neuropod, Uber ATG’s Open Source Deep Learning Inference Engine by Vivek Panyam 375 days ago.
	It's time to rethink how we share data on the Web by Daniel Abadi 548 days ago.
	Introducing SLOG: Cheating the low-latency vs. strict serializability tradeoff by Daniel Abadi 561 days ago.
	The dangers of conditional consistency guarantees by Daniel Abadi 561 days ago.
	Introduction to Transaction Isolation Levels by Daniel Abadi 561 days ago.
	Correctness Anomalies Under Serializable Isolation by Daniel Abadi 561 days ago.
	Overview of Consistency Levels in Database Systems by Daniel Abadi 561 days ago.
	An explanation of the difference between Isolation levels vs. Consistency levels by Daniel Abadi 561 days ago.
	Partitioned consensus and its impact on Spanner’s latency by Daniel Abadi 562 days ago.
	It’s Time to Move on from Two Phase Commit by Daniel Abadi 562 days ago.
	NewSQL database systems are failing to guarantee consistency, and I blame Spanner by Daniel Abadi 562 days ago.
	An analysis of the strengths and weaknesses of Apache Arrow by Daniel Abadi 1179 days ago.
	Apache Arrow vs. Parquet and ORC: Do we really need a third Apache project for columnar data representation? by Daniel Abadi 1326 days ago.
	Hazelcast and the Mythical PA/EC System by Daniel Abadi 1349 days ago.
	Scala Days 2017 - Jepsen Keynote by Aphyr 1507 days ago.
	Distributed consistency at scale: Spanner vs. Calvin by Daniel Abadi 1534 days ago.
	Jepsen: CockroachDB beta-20160829 by Aphyr 1583 days ago.
	Jepsen: MongoDB 3.4.0-rc3 by Aphyr 1592 days ago.
	IEEE Computer issue on the CAP Theorem by Daniel Abadi 1769 days ago.
	Jepsen: VoltDB 6.3 by Aphyr 1802 days ago.
	Jepsen: Crate 0.54.9 version divergence by Aphyr 1816 days ago.
	Jepsen: RethinkDB 2.2.3 reconfiguration by Aphyr 1974 days ago.
	Jepsen: RethinkDB 2.1.5 by Aphyr 1992 days ago.
	Why MongoDB, Cassandra, HBase, DynamoDB, and Riak will only let you perform transactions on a single data item by Daniel Abadi 2058 days ago.
	Jepsen: Percona XtraDB Cluster by Aphyr 2114 days ago.
	Jepsen: MariaDB Galera Cluster by Aphyr 2117 days ago.
	Jepsen: Chronos by Aphyr 2139 days ago.
	Jepsen: Aerospike by Aphyr 2237 days ago.
	Jepsen: Elasticsearch 1.5.0 by Aphyr 2244 days ago.
	Jepsen: MongoDB stale reads by Aphyr 2251 days ago.
	Jepsen: Elasticsearch by Aphyr 2560 days ago.
	Jepsen: etcd and Consul by Aphyr 2566 days ago.
	Jepsen: RabbitMQ by Aphyr 2570 days ago.
	Computational techniques in Knossos by Aphyr 2589 days ago.
	Strong consistency models by Aphyr 2592 days ago.
	Jepsen: Redis redux by Aphyr 2747 days ago.
	Jepsen: Strangeloop Hangout by Aphyr 2820 days ago.
	Jepsen: RICON East talk by Aphyr 2820 days ago.
	Jepsen: Cassandra by Aphyr 2824 days ago.
	Jepsen: Kafka by Aphyr 2825 days ago.
	Jepsen: NuoDB by Aphyr 2826 days ago.
	Jepsen: Zookeeper by Aphyr 2826 days ago.
	Automating Jepsen by Aphyr 2912 days ago.
	The network is reliable by Aphyr 2938 days ago.
	Asynchronous replication with failover by Aphyr 2950 days ago.
	Jepsen: final thoughts by Aphyr 2952 days ago.
	Jepsen: Riak by Aphyr 2952 days ago.
	Jepsen: MongoDB by Aphyr 2953 days ago.
	Jepsen: On the perils of network partitions by Aphyr 2953 days ago.
	Jepsen: Redis by Aphyr 2953 days ago.
	Jepsen: Postgres by Aphyr 2953 days ago.
	Is Upstart the right way to get college student start-ups funded? by Daniel Abadi 3145 days ago.
	Defending Matt Welsh’s 'Startup University' Post by Daniel Abadi 3279 days ago.
	If all these new DBMS technologies are so scalable, why are Oracle and DB2 still on top of TPC-C? A roadmap to end their dominance. by Daniel Abadi 3320 days ago.
	Replication and the latency-consistency tradeoff by Daniel Abadi 3480 days ago.
	Overview of the Oracle NoSQL Database by Daniel Abadi 3542 days ago.
	Hadoop's tremendous inefficiency on graph data management (and how to avoid it) by Daniel Abadi 3613 days ago.
	Why Sam Madden is wrong about peer review by Daniel Abadi 3683 days ago.
	Why I'm doing a start-up pre-tenure by Daniel Abadi 3734 days ago.

Data Engineering

All things related to data engineering - databases, distributed systems, big data and much more

Sources

Updated at
Sat Jun 19 12:13:02