Senior Data Engineer – Data Platform & Databricks Expert

Who We Are

Join a team that puts its People First! As a member of First American's family of companies, National Commercial Services provides single-point service for simple to multi-property/multi-state national commercial real estate transactions. Since 1889, First American (NYSE: FAF) has held an unwavering belief in its people. They are passionate about what they do, and we are equally passionate about fostering an environment where all feel welcome, supported, and empowered to be innovative and reach their full potential. Our inclusive, people-first culture has earned our company numerous accolades, including being named to the Fortune 100 Best Companies to Work For® list for ten consecutive years. We have also earned awards as a best place to work for women, diversity and LGBTQ+ employees, and have been included on more than 50 regional best places to work lists. First American will always strive to be a great place to work, for all. For more information, please visit www.careers.firstam.com.

What We Do

We are seeking a Senior Data Engineer with deep expertise in Databricks and a proven track record in modern data platform architecture, ETL implementation, and best practices. The successful candidate will play a crucial role in designing, building, and optimizing our big data solutions, with Databricks as our core platform. Additionally, you will contribute to shaping our overall data strategy and driving innovation.

Key Responsibilities:

• Databricks Subject Matter Expert (SME): Serve as Databricks Subject Matter Expert (SME): Lead the adoption, optimization, and evangelization of Databricks and its ecosystem (Delta Lake, Unity Catalog, MLflow, Databricks SQL).
• Big Data Solution Architecture: Architect, build, and optimize big data solutions using Databricks and other modern data platforms such as Azure SQL, Synapse, and Snowflake.
• Data Strategy and Governance: Drive our data strategy, ensuring alignment with business goals and best practices. Lead platform governance and ensure compliance with industry standards.
• Performance Optimization: Continuously optimize Databricks clusters, jobs, and workflows for performance, reliability, and cost-efficiency. Apply similar optimization principles across adjacent data platforms as needed.
• Best Practices and Mentorship: Lead the development and implementation of best practices for coding, data management, and platform operations. Mentor and upskill engineers, data scientists, and analysts on Databricks features and data engineering best practices.
• Innovation and Industry Engagement: Stay updated on the latest industry trends and innovations in data engineering. Engage in Databricks forums, conferences, and user groups to share knowledge and learn from the community.
• Incident Management: Lead incident response efforts, troubleshoot complex issues, and implement long-term solutions.

What You'll Bring:

• At least 7 years of experience in data engineering, with a minimum of 3 years in architecting and operating Databricks solutions in production environments.
• Expert-level proficiency in Databricks, including Delta Lake and Unity Catalog.
• Must have experience implementing and optimizing ETL/ELT on modern platforms such as Azure SQL, Synapse, Databricks, and Snowflake, with a demonstrated ability to integrate multiple data sources (including structured and unstructured data, APIs, and streams). Expertise in Delta Lake and Apache Iceberg formats for scalable batch and near real-time processing is required.
• Strong programming skills in Python and PySpark with experience developing production-grade data processing applications and optimizing Spark jobs for performance.
• Strong proficiency in SQL and modern data modeling methods (including medallion architecture, dimensional modeling, and star/snowflake schemas), with the capability to design efficient data structures for analytics.
• Preferred: Experience building CI/CD pipelines for data platforms using Databricks CLI, Terraform for infrastructure as code, and Git-based workflows to automate testing and deployment of data solutions.
• Preferred: Practical experience with machine learning workflows and MLOps practices, such as model training, versioning, and deployment using Databricks ML tools and MLflow.
• Demonstrated ability to lead teams and mentor junior engineers and data scientists.
• Experience in leading migrations from legacy platforms to modern data platforms, including Databricks.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent experience).

Bonus Points:

• Databricks certifications (e.g., Databricks Certified Data Engineer Professional).
• Experience with real-time streaming technologies (e.g., Structured Streaming, Kafka).
• Contributions to Databricks forums, conferences, or other industry communities.

Pay Range: $126,100-$168,100 annually

This hiring range is a reasonable estimate of the base pay range for this position at the time of posting.  Pay is based on a number of factors which may include job-related knowledge, skills, experience, business requirements and geographic location.

What We Offer

By choice, we don’t simply accept individuality – we embrace it, we support it, and we thrive on it! Our People First Culture celebrates diversity, equity and inclusion not simply because it’s the right thing to do, but also because it’s the key to our success. We are proud to foster an authentic and inclusive workplace For All. You are free and encouraged to bring your entire, unique self to work. First American is an equal opportunity employer in every sense of the term.

Based on eligibility, First American offers a comprehensive benefits package including medical, dental, vision, 401k, PTO/paid sick leave and other great benefits like an employee stock purchase plan.