5+ YoE

Candidates ready to join immediately can share their details via email for quick processing.

CCTC | ECTC | Notice Period | Location Preference

Act fast for immediate attention

5+ years if Experience

Roles and Responsibilities

• Design, develop, and maintain scalable data pipelines using Spark (PySpark or Spark with Scala).
• Build data ingestion and transformation frameworks for structured and unstructured data sources.
• Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions.
• Work with large volumes of data and ensure quality, integrity, and consistency.
• Optimize data workflows for performance, scalability, and cost efficiency on cloud platforms (AWS, Azure, or GCP).
• Implement data quality checks and automation for ETL/ELT pipelines.
• Monitor and troubleshoot data issues in production and perform root cause analysis.
• Document technical processes, system designs, and operational procedures.

Must-Have Skills

• 3+ years of experience as a Data Engineer or similar role.
• Hands-on experience with PySpark or Spark using Scala .
• Strong knowledge of SQL for data querying and transformation.
• Experience working with any cloud platform (AWS, Azure, or GCP).
• Solid understanding of data warehousing concepts and big data architecture.
• Experience with version control systems like Git.

Good-to-Have Skills

• Experience with data orchestration tools like Apache Airflow , Databricks Workflows , or similar.
• Knowledge of Delta Lake , HDFS , or Kafka .
• Familiarity with containerization tools (Docker/Kubernetes).
• Exposure to CI/CD practices and DevOps principles.
• Understanding of data governance, security, and compliance standards.

.