Farah Siddiqui Data Engineer
San Francisco, CA • farah.siddiqui@gmail.com • +1 415-555-0163
Profile Summary
- Data Engineer with 6 years of experience building batch and streaming data platforms for consumer banking and rideshare apps with millions of users, focused on dimensional modeling, change data capture, and data quality.
- Hands-on with languages (Python, SQL), processing (Apache Spark, dbt), streaming (Apache Kafka, Apache Flink), orchestration and ingestion (Apache Airflow, Fivetran), warehouses and table formats (Snowflake, Apache Iceberg), and cloud and infrastructure (AWS, Terraform), backed by a solid grasp of query plans, distributed systems, and data structures.
- Strong in star schemas with slowly changing dimensions, incremental ELT, CDC with Debezium, and Iceberg table design, using schema contracts with source teams and freshness SLAs with automated alerts to deliver tables that analysts and ML models can rely on every morning.
- Works every day with analysts, ML engineers, and the backend teams that own source systems in two-week sprints with a shared on-call rotation, from scoping a new dataset with the people who will query it to the fixes after it ships.
- Mentors newer engineers on SQL performance and testing dbt models through code review and pairing, leads the monthly data platform design review, and wrote the dbt style guide every new hire on the data team follows.
Technical Skills
- Languages:
- Python, SQL, Scala, Bash
- Processing & Transformation:
- Apache Spark, dbt, PySpark, Trino, pandas
- Streaming & Messaging:
- Apache Kafka, Apache Flink, Debezium, Kafka Connect, Amazon Kinesis
- Orchestration & Ingestion:
- Apache Airflow, Fivetran, Dagster, Airbyte
- Warehouses & Lakehouses:
- Snowflake, Apache Iceberg, BigQuery, Delta Lake, PostgreSQL
- Cloud & Infrastructure:
- AWS, Terraform, Docker, Kubernetes, GitHub Actions
- Data Quality & Observability:
- dbt tests, Great Expectations, Monte Carlo, OpenLineage, Datadog
Education
Work Experience
- Built 140 ELT pipelines with Fivetran and Python connectors that load card transactions, support tickets and partner bank files into Snowflake, with incremental loads and idempotent reruns, cutting the time to add a new source from 2 weeks to 3 days.
- Designed the member and transactions marts in dbt as a star schema with Type 2 slowly changing dimensions, replacing 310 overlapping SQL views with one model that 180 analysts in finance, risk and growth now query.
- Moved 85 cron and Lambda jobs onto Airflow with data-aware scheduling, sensors, retries and one-command backfills, raising on-time delivery of the 7am finance tables from 86% to 99.6%.
- Built change data capture from the transactions Postgres database with Debezium, streamed through Kafka and processed in Flink with exactly-once checkpoints, cutting the lag on fraud model features from 4 hours to 40 seconds.
- Put Great Expectations checks and Monte Carlo freshness SLAs on 210 critical tables and masked PII columns with tag-based policies, so 93% of data incidents were caught before anyone saw them on a dashboard.
- Cut monthly Snowflake spend by 34% by clustering the largest fact tables, turning full refreshes into incremental dbt models, and right-sizing warehouses with auto-suspend and per-team cost tags.
- Moved the data platform to Terraform modules and GitHub Actions pipelines that lint SQL, build dbt models against staging and test Airflow DAGs on every pull request, cutting failed production deploys from 9 a month to 1.
- Agreed on schema contracts with the payments and card engineering teams that own the source services and ran monthly data reviews with the analytics and ML teams, cutting pipeline breaks from upstream schema changes from 11 a quarter to 2.
- Built PySpark jobs on Amazon EMR that turn 900 million ride events a day into hourly marketplace tables for the pricing team, cutting the nightly batch runtime from 5.5 hours to 1.8 hours.
- Documented 420 core tables in Amundsen, the company’s data catalog, with owners, descriptions and lineage, which cut data questions in the analytics Slack channel by 35%.
- Moved driver incentive reporting from nightly Hive jobs to Flink streaming jobs on Kafka, so city operations teams could see incentive spend within 15 minutes instead of the next morning.