热点^

Ep2. Data Engineering at Uber and Lyft

2021-01-25 09:35:10 117

声音简介

这一期节目我们和嘉宾泉来聊一聊Data Engineering，在Uber和Lyft的应用和工作体验，对于行业发展的反思与展望，以及对于入行Data Engineering的建议。只要你对Large Scale Data Engineering感兴趣，本期节目都会对你有帮助 :)

---

Timestamps:
    00:00:00 Intro
    00:00:40 Data Infrastructure vs Data Engineer
    00:02:45 Data Engineering 领域介绍
    00:09:00 Pipeline权限设置
    00:11:00 Pipeline在Uber的使用场景
    00:11:45 如何追踪Data Owner
    00:11:52 如何保护用户数据隐私
    00:17:16 Data Infra转到Data Engineer的日常工作
    00:21:44 Data Quality Tier
    00:32:52 Uber vs Lyft 数据量级和迭代的区别
    00:35:46 COVID影响
    00:38:40 对颠覆性产业和商业模式的反思
    00:40:27 规模效应和数据
    00:41:26 ML Infra vs Data Infra
    00:44:47 对新入行Data Engineering的建议
    00:47:28 联系方式

Links:
    Airflow: https://airflow.apache.org/
    Piper: https://eng.uber.com/no-code-workflow-orchestrator/
    uWorc: https://eng.uber.com/no-code-workflow-orchestrator/

    Amundsen:https://eng.lyft.com/open-sourcing-amundsen-a-data-discovery-and-metadata-platform-2282bb436234
    PII: https://en.wikipedia.org/wiki/Personal_data
    GDPR: https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
    ETL: https://en.wikipedia.org/wiki/Extract,_transform,_load
    联系泉来：mail@quanlai.li

上一个：Ep3. 算法题和科技公司面试的那些事

下一个： Ep1. 职场One on one谈些啥