Middle Data Engineer (Apache Spark, Trino)

IMIP Technology And Solution Consultancy • Tầng 5,Tòa nhà Lucky,số 81 Trần Thái Tông,Cau Giay,Ha Noi

Trách nhiệm chính

Design, build, and maintain automated data ingestion pipelines from multiple source systems using Airbyte and Airflow.
Develop and manage the Medallion Lakehouse Architecture (Bronze/Silver/Gold) with dbt‑spark to produce high‑quality analytical datasets.
Optimize Spark and Trino workloads on Kubernetes, managing the full lifecycle of Apache Iceberg tables to improve query performance and reduce latency.
Proactively monitor and maintain Data Platform resources with Grafana and Prometheus, analyzing query patterns to enhance system stability.
Enforce row‑ and column‑level security using Open Policy Agent and build foundational frameworks for Data Governance and Data Quality.
Architect and implement high‑performance data bridges to deliver on‑premise Lakehouse data into Microsoft Fabric for seamless hybrid‑cloud connectivity.

Yêu cầu công việc

3+ years of hands‑on experience in Data Engineering or Platform Engineering, building and operating production‑grade data platforms.
Strong expertise in distributed processing with Apache Spark and Trino, including performance tuning and optimization.
Proven experience designing and maintaining reliable data pipelines using Airbyte and Apache Airflow.
Solid understanding of modern Lakehouse architectures, dbt‑based transformations, and Iceberg table management.
Hands‑on experience with Kubernetes, containerized deployments, and CI/CD for data workloads.
Knowledge of data security, governance, and access control practices (row/column‑level security, policy enforcement, IAM).
Experience supporting BI/analytics use cases and integrating with enterprise reporting tools such as Microsoft Fabric and Power BI.
Strong troubleshooting, system reliability, and problem‑solving skills in complex distributed environments.
Ability to work independently in a highly technical, ownership‑driven role.

Mức lương

Phúc lợi

Thông tin khác

Middle Data Engineer (Apache Spark, Trino)