Big Data Lead Engineer — Hadoop, Spark

Job Summary

Synechron is seeking an experienced Big Data Lead Engineer to drive the design, development, and management of large-scale data solutions. This full-time leadership role involves guiding a team of data engineers and architects to deliver scalable, efficient, and innovative big data initiatives aligned with organizational and business objectives. The role aims to enable data-driven decision-making, optimize data processing workflows, and stay ahead of emerging industry trends to enhance the company's technological capabilities.

Software Requirements

Candidates must possess strong proficiency with big data technologies such as Hadoop, Spark, and NoSQL databases, with proven experience deploying and managing these tools in enterprise environments. Familiarity with data warehousing, ETL processes, and data visualization tools is required. Hands-on experience with cloud platforms, specifically Google Cloud Platform (GCP), is essential, including managing cloud-based data processing tools like DataProc and BigQuery.

Preferred skills include experience with data pipeline orchestration tools, automation frameworks, and advanced analytics solutions. Knowledge of scripting languages such as Python, shell scripting, and familiarity with security and data governance practices will be advantageous.

Overall Responsibilities

Lead a team of big data developers and architects to deliver high-quality data solutions, providing technical guidance, mentorship, and support.

Develop and execute a comprehensive big data strategy that supports business goals, ensuring alignment with organizational priorities.

Oversee project delivery, ensuring timely, within-budget execution that meets or exceeds stakeholder expectations.

Design, implement, and optimize scalable big data architectures leveraging Hadoop, Spark, and NoSQL technologies.

Collaborate with cross-functional teams including product, technology, and marketing to gather requirements and translate them into technical solutions.

Stay current with evolving big data trends and emerging technologies, assessing their applicability to improve existing workflows.

Manage multiple projects, balancing stakeholder needs with team capacity, and prioritize efforts to maximize impact.

Foster a culture of continuous learning, professional growth, and innovation within the team.

Technical Skills (By Category)

Programming Languages:
Required: Proficiency in Java and Python for developing applications, workflows, and automations.
Preferred: Knowledge of scripting languages such as Shell or Perl to support automation and deployment tasks.

Databases/Data Management:
Extensive experience with Hadoop ecosystem components (HDFS, Hive, Spark) and NoSQL databases like Cassandra or MongoDB. Strong capabilities with cloud data warehouses such as BigQuery and Cloud SQL.

Cloud Technologies:
Deep understanding of Google Cloud Platform (GCP), including DataProc, BigQuery, Cloud Storage, and related cloud services for data management and processing.

Frameworks and Libraries:
Hands-on experience with Apache Spark, Hive, and other big data processing libraries; familiarity with data serialization formats like Parquet or Avro.

Development Tools and Methodologies:
Expertise in version control tools such as Git, Agile development practices, and automated CI/CD pipelines to promote efficient workflow management.

Security Protocols:
Knowledge of data security, access control, and compliance standards relevant to cloud data environments to ensure data protection.

Experience Requirements

Minimum of 7+ years of experience in big data and related software development roles.

Real-world leadership experience managing teams of data engineers and architects for at least 5 years.

Proven track record delivering large-scale big data projects within enterprise or consulting environments.

Strong background in designing scalable architectures with Hadoop, Spark, and NoSQL databases.

Industry experience in finance, healthcare, retail, or technology sectors is preferred but not mandatory.

Alternative candidates may demonstrate extensive hands-on experience in data engineering, cloud data migrations, or enterprise analytics projects.

Day-to-Day Activities

Lead design and implementation of big data solutions, including architecture development and data pipeline creation.

Mentor team members through code reviews, technical mentoring, and knowledge-sharing sessions.

Collaborate with stakeholders to clarify requirements, define technical specifications, and ensure solutions meet business needs.

Monitor project progress, manage workload, and address technical challenges promptly.

Ensure data solutions are optimized for performance, reliability, and security.

Stay updated on industry trends and evaluate new tools or processes that could improve project outcomes.

Coordinate across multidisciplinary teams to achieve project milestones and deliverables.

Document technical architecture, processes, and best practices for team capacity building and future reference.

Qualifications

Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related technical discipline.

Certifications in big data technologies (e.g., Cloudera Certified Professional, Databricks, or GCP certifications) are preferred.

Proven experience designing and delivering large enterprise data solutions.

Strong commitment to continuous learning and professional development.

Excellent problem-solving, analytical, and troubleshooting skills.

Professional Competencies

Critical thinking to analyze complex data challenges and develop innovative solutions.

Leadership qualities to inspire, mentor, and develop technical talent.

Effective communication skills, capable of explaining complicated technical concepts to non-technical stakeholders.

Strong collaboration and stakeholder management abilities.

Adaptability to rapidly evolving technological trends and project demands.

Ability to manage multiple priorities and deliver results within deadlines.

Forward-thinking mindset focused on sustainable, scalable, and efficient analytics solutions.

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.