Spark Data Engineer

1 week ago

Mumbai, Maharashtra, India Mactores Full time ₹ 12,00,000 - ₹ 36,00,000 per year
Mactores is a trusted leader among businesses in providing modern data platform solutions. Since 2008, Mactores have been enabling businesses to accelerate their value through automation by providing End-to-End Data Solutions that are automated, agile, and secure. We collaborate with customers to strategize, navigate, and accelerate an ideal path forward with a digital transformation via assessments, migration, or modernization.

We are seeking a highly skilled and innovative Spark Engineer to join our team. In this role, you will design, develop, optimize, and operationalize high-performance data pipelines and applications using Apache Spark. This role requires hands-on expertise in distributed data processing, ETL engineering, performance tuning, cluster management, and working with cross-functional teams to deliver reliable, scalable, and efficient data solutions What will you do
  • Architect, design, and build scalable data pipelines and distributed applications using Apache Spark (Spark SQL, DataFrames, RDDs)
  • Develop and manage ETL/ELT pipelines to process structured and unstructured data at scale.
  • Write high-performance code in Scala or PySpark for distributed data processing workloads.
  • Optimize Spark jobs by tuning shuffle, caching, partitioning, memory, executor cores, and cluster resource allocation.
  • Monitor and troubleshoot Spark job failures, cluster performance, bottlenecks, and degraded workloads.
  • Debug production issues using logs, metrics, and execution plans to maintain SLA-driven pipeline reliability.
  • Deploy and manage Spark applications on on-prem or cloud platforms (AWS, Azure, or GCP).
  • Collaborate with data scientists, analysts, and engineers to design data models and enable self-serve analytics.
  • Implement best practices around data quality, data reliability, security, and observability.
  • Support cluster provisioning, configuration, and workload optimization on platforms like Kubernetes, YARN, or EMR/Databricks.
  • Maintain version-controlled codebases, CI/CD pipelines, and deployment automation.
  • Document architecture, data flows, pipelines, and runbooks for operational excellence
What we are looking for
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • 4+ years of experience building distributed data processing pipelines, with deep expertise in Apache Spark.
  • Strong understanding of Spark internals (Catalyst optimizer, DAG scheduling, shuffle, partitioning, caching).
  • Proficiency in Scala and/or PySpark with strong software engineering fundamentals.
  • Solid expertise in ETL/ELT, distributed computing, and large-scale data processing.
  • Experience with cluster and job orchestration frameworks.
  • Strong ability to identify and resolve performance bottlenecks and production issues.
  • Familiarity with data security, governance, and data quality frameworks.
  • Excellent communication and collaboration skills to work with distributed engineering teams.
  • Ability to work independently and deliver scalable solutions in a fast-paced environment
You will be preferred if
  • Experience with Databricks, AWS EMR, Glue Spark, or GCP Dataproc.
  • Familiarity with workflow orchestration tools like Apache Airflow, Dagster, or Prefect.
  • Exposure to streaming platforms such as Kafka, Kinesis, or Pub/Sub.
  • Experience running Spark workloads on Kubernetes.
  • Familiarity with data warehouse ecosystems (Snowflake, BigQuery, Redshift, Iceberg, Delta Lake, Hudi).
  • Understanding of DevOps practices, CI/CD, and IaC (Terraform, CloudFormation).
  • Knowledge of distributed logging and monitoring tools (Grafana, Prometheus, CloudWatch, ELK).
  • Prior experience in high-scale production environments or data platform teams
Life at Mactores
We care about creating a culture that makes a real difference in the lives of every Mactorian. Our 10 Core Leadership Principles that honor Decision-making, Leadership, Collaboration, and Curiosity drive how we work.
1. Be one step ahead 2. Deliver the best 3. Be bold 4. Pay attention to the detail 5. Enjoy the challenge 6. Be curious and take action 7. Take leadership 8. Own it 9. Deliver value 10. Be collaborative
We would like you to read more details about the work culture on  
The Path to Joining the Mactores Team At Mactores, our recruitment process is structured around three distinct stages:
Pre-Employment Assessment:  You will be invited to participate in a series of pre-employment evaluations to assess your technical proficiency and suitability for the role.
Managerial Interview: The hiring manager will engage with you in multiple discussions, lasting anywhere from 30 minutes to an hour, to assess your technical skills, hands-on experience, leadership potential, and communication abilities.
HR Discussion: During this 30-minute session, you'll have the opportunity to discuss the offer and next steps with a member of the HR team.
At Mactores, we are committed to providing equal opportunities in all of our employment practices, and we do not discriminate based on race, religion, gender, national origin, age, disability, marital status, military status, genetic information, or any other category protected by federal, state, and local laws. This policy extends to all aspects of the employment relationship, including recruitment, compensation, promotions, transfers, disciplinary action, layoff, training, and social and recreational programs. All employment decisions will be made in compliance with these principles.

Note: Please answer as many questions as possible with this application to accelerate the hiring process. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

  • Spark Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Mactores Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Mactores is a trusted leader among businesses in providing modern data platform solutions. Since 2008, Mactores have been enabling businesses to accelerate their value through automation by providing End-to-End Data Solutions that are automated, agile, and secure. We collaborate with customers to strategize, navigate, and accelerate an ideal path forward...

  • Python Spark Developer

    1 day ago

    Mumbai, Maharashtra, India Synechron Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummarySynechron is seeking a skilled Python Spark Developer to design and optimize large-scale data pipelines and processing systems. The successful candidate will leverage expertise in Python and Apache Spark to build scalable, high-performance data workflows, supporting enterprise analytics, fraud detection, and real-time data applications. This role...

  • Software Engineer III Python, Spark,Data pipelines

    3 days ago

    Mumbai, Maharashtra, India JPMorganChase Full time ₹ 60,000 - ₹ 1,20,000 per year

    DescriptionBe part of a dynamic team where your distinctive skills will contribute to a winning culture and team.  As a Data Engineer III at JPMorgan Chase within the Corporate Technology, you serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable...

  • Software Engineer III Python, Spark,Data pipelines

    3 days ago

    Mumbai, Maharashtra, India JPMorganChase Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    JOB DESCRIPTIONBe part of a dynamic team where your distinctive skills will contribute to a winning culture and team.As a Data Engineer III at JPMorgan Chase within the Corporate Technology, you serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable...

  • L2 Application Support Engineer

    1 day ago

    Mumbai, Maharashtra, India Optimum Data Analytics Full time ₹ 10,00,000 - ₹ 12,00,000 per year

    • Perform 24×7 monitoring of Databricks clusters, jobs, workflows, repos, and data pipelines.• Alert Monitoring and first level of resolution thereof• First Level issue troubleshooting/analysis related to ✓ Cluster failures or auto-scaling issues ✓ Job failures(PySpark/Scala/Spark SQL/Delta Live Tables) ✓ Workspace availability issues• Debug...

  • Senior Data Engineer

    4 days ago

    Mumbai, Maharashtra, India scymes services pvt limited Full time ₹ 10,00,000 - ₹ 15,00,000 per year

    Educational Background: Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).Experience: ○ 6+ years of experience as a Data Engineer, with proficiency in at least two major cloud platforms (AWS, Azure, GCP). ○Proven experience in designing, developing, and implementing comprehensive data engineering...

  • Data Engineer

    1 day ago

    Mumbai, Maharashtra, India Growel Softech Pvt. Ltd. Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    :We are looking for a skilled Data Engineer/Developer with expertise in Snowflake and Spark as primary skills. The ideal candidate should also have a strong understanding of SQL and Hadoop.Primary Skills: Snowflake Spark Secondary Skills:SQL Hadoop Other Requirement:Strong communication skillsAdditional DetailsGlobal Grade : CRemote work possibility :...

  • Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Hire22 Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    One of Our Client Hiring a Data Engineer with 5 to 10 years of experience skilled in Databricks, Spark, ETL, and cloud platforms.The role involves building scalable data pipelines, optimising workflows, and managing end-to-end data processing across AWS, Azure, or GCP. Location is Navi Mumbai or Pune.Key Responsibilities:Design, develop, and maintain data...

  • Remote Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Go Digital Technology Consulting Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Designation: Data Engineer (Engineer & Senior Engineer Level)Location: Remote (across India)Experience: 3 to 8 years Technologies / Skills:Strong hands-on experience with AWS data engineering services (ETL, orchestration, and streaming tools.Proficiency in SQL, Python (Pandas, NumPy) and PySpark.Experience in ETL/ELT pipeline development, data modeling and...

  • Scala Spark Developer

    3 days ago

    Navi Mumbai, Maharashtra, India Capgemini Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...