Data Engineer (Databricks/Azure)
About This Job
Trách nhiệm chính
- Design, develop, and maintain scalable data pipelines on Databricks and Microsoft Azure.
- Build Lakeflow Declarative Pipelines to support reliable and automated data processing.
- Develop production‑grade data processing solutions using Python, PySpark, and SQL.
- Optimize Databricks jobs, clusters, execution plans, partitioning strategies, shuffling, and memory usage.
- Implement Lakehouse design patterns, data modeling approaches, and data warehousing structures.
- Manage data governance and access controls using Unity Catalog.
- Build and maintain CI/CD pipelines with Azure DevOps and GitFlow branching practices.
- Implement infrastructure as code using Terraform and support metadata‑driven ingestion patterns, including REST API integrations.
Yêu cầu
- Bachelor's Degree in Data Engineering, Computer Science, IT or related field.
- 3+ years of experience in data engineering or similar role.
- Proficient with Databricks, including Lakehouse architecture, workspace administration, access management, cluster configuration, and cost optimization.
- Hands‑on experience developing Lakeflow Declarative Pipelines.
- Strong programming expertise in Python and PySpark for scalable, production‑grade data processing.
- Advanced proficiency in SQL, including complex transformations and performance tuning.
- Strong experience with Terraform for infrastructure as code.
- Proven experience designing and maintaining CI/CD pipelines.
- Solid hands‑on experience with Microsoft Azure and Azure DevOps.
- Experience with data modeling, data warehousing, Lakehouse design patterns, metadata‑driven pipelines, REST API integration, and GitFlow.
- Good communication in English.
- Strong analytical mindset, problem‑solving skills, attention to detail, and ability to troubleshoot complex data platform issues.
Phúc lợi
- Competitive salary and 13th‑month salary
- 14+ annual leaves per year
- Premium healthcare insurance, starting from your probation period
- Project reviews and yearly performance appraisals
- Annual company trip
- Teambuilding activities: Team lunch/dinner, events, and celebrations, sports clubs (football, basketball, badminton, pickleball)
- International team environment
- Tailor‑made career path
- Technical workshops and training courses
- Mobility: Opportunities to be on‑site abroad in our offices in over 60+ countries
Thông tin khác
- Equal Opportunity workplace: Amaris Consulting welcomes applications from all qualified candidates regardless of gender, sexual orientation, race, ethnicity, beliefs, age, marital status, disability, or other characteristics.
- Recruitment process includes a brief virtual/phone conversation, interviews (average 3), and a case study or technical assessment.