Lead Big Data Developer

8 hours ago

Chennai, Tamil Nadu, India Ciklum Full time

About Ciklum

Ciklum is a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts, and product owners, we engineer technology that redefines industries and shapes the way people live.

Job Description

We are looking for an experienced Lead Big Data Developer to join our team full-time in India. As a key member of our cross-functional development team, you will be responsible for designing, building, deploying, and maintaining mission-critical analytics solutions that process data quickly at big data scales.

Responsibilities

  1. Design and implement scalable and efficient big data architectures using technologies such as Hadoop, MapReduce, Pig, Hive, Impala, Spark, Kafka, Storm, SQL, and NoSQL data warehouses like HBase and Cassandra.
  2. Collaborate with data scientists and engineers to develop innovative data processing pipelines and workflows.
  3. Contribute to the development of cloud-based data platforms using AWS, Azure, or GCP.
  4. Own components of the engineering infrastructure and work to continually improve it, identifying gaps and improving the platform's quality, robustness, maintainability, and speed.
  5. Provide technical leadership and guidance to junior team members, mentoring them on best practices and new technologies.
  6. Interact with engineering teams and ensure that solutions meet customer requirements in terms of functionality, performance, availability, scalability, and reliability.
  7. Perform development, QA, and dev-ops roles as needed to ensure total end-to-end responsibility of solutions.
  8. Work directly with business analysts and data scientists to understand and support their use cases.
  9. Participate in Agile software development methodologies and contribute to the success of the team.

Requirements

  1. 5+ years of experience coding in SQL, Java, Python, Scala, with solid CS fundamentals including data structure and algorithm design.
  2. 3+ years contributing to production deployments of large backend data processing and analysis systems as a team lead.
  3. 2+ years of hands-on implementation experience working with a combination of the following technologies: Hadoop, MapReduce, Pig, Hive, Impala, Spark, Kafka, Storm, SQL, and NoSQL data warehouses like HBase and Cassandra.
  4. 3+ years of experience in cloud data platforms (AWS, Azure, GCP).
  5. Detailed knowledge and understanding of Data Governance process definition, development, and deployment.
  6. Demonstrable experience in leading development teams, including managing the team backlog and assigning tasks to team members.
  7. Ability to work on and manage multiple projects simultaneously while adapting to changing priorities in a fast-paced environment.
  8. Knowledge of professional software engineering best practices for the full software life cycle.

Benefits

  • A competitive salary of ₹1,200,000 - ₹1,500,000 per annum, depending on experience.
  • A comprehensive benefits package, including medical insurance, financial consultation, and language courses.
  • A flexible hybrid work mode at Chennai or Pune, allowing for remote work options.
  • An opportunity to work on large-scale projects that redefine industries with international clients.
  • A welcoming environment with a friendly team, open-door policy, and regular team-building events.

  • Big Data Lead

    1 month ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Description:As a Big Data Lead at Virtusa, you will be responsible for designing, building, and supporting data applications using Spark, Sqoop, and Hive. You will work with large data sets and high-capacity big data processing platforms, leveraging your expertise in Unix and Shell scripting. Your advanced knowledge of the Hadoop ecosystem and its...

  • Big Data Lead

    1 month ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Title: Big Data LeadJob Summary:We are seeking a highly skilled Big Data Lead to join our team at Virtusa. The successful candidate will have a strong background in designing, building, and supporting data applications using Spark, Sqoop, and Hive.Key Responsibilities:Develop components, application interfaces, and solution enablers while ensuring...

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    Job Title: Big Data DeveloperJob Summary:RiskInsight Consulting Pvt Ltd is seeking a skilled Big Data Developer to join our team. As a Big Data Developer, you will design, develop, and deploy scalable Big Data applications using Apache Spark.Responsibilities:Data Ingestion: Implement data ingestion processes from various data sources to the Big Data...

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    Job Title: Big Data DeveloperAbout the Role:RiskInsight Consulting Pvt Ltd is seeking a skilled Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and deploying scalable Big Data applications using Hadoop ecosystem technologies.Key Responsibilities:Design and develop scalable Big Data applications...

  • Big Data Developer

    2 months ago

    Chennai, Tamil Nadu, India Mavinsys Full time

    Minimum Required Skills & Experience :1. Big Data Technologies (6+ years) :- Extensive experience with Big Data frameworks like Hadoop and Spark.- Deep understanding of Hadoop ecosystem (HDFS, YARN, MapReduce).- Familiarity with big data schedulers (e.g., Oozie, Airflow).2. Scala & Spark Expertise (4+ years each) :- Strong proficiency in Scala programming...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job DescriptionAs a Data Architecture Specialist at Virtusa, you will have the opportunity to design and build high-performance big data applications using Spark, Sqoop, and Hive.We are seeking a highly experienced and skilled professional with a strong background in big data processing and a bachelor's or master's degree in Computer Science or a related...

  • Big Data Platform Technical Lead

    3 weeks ago

    Chennai, Tamil Nadu, India 1HResource Solutions Full time

    Job Title: Senior Data Engineer for Large-Scale Data IntegrationJob Description: We are seeking an experienced Senior Data Engineer to lead our big data platform initiatives. The successful candidate will be responsible for designing and implementing large-scale data integration projects using big data technologies such as Hadoop, Spark, and Kafka.Key...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Job Title: Big Data DeveloperDescription:TCS is hiring a Big Data Developer to work on our exciting projects. As a Big Data Developer, you will be responsible for transferring and transforming data with Azure Synapse Analytics pipelines.Responsibilities:Transfer and transform data with Azure Synapse Analytics pipelines.Build a data pipeline in Azure Synapse...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Snaphunt Full time

    The OpportunitySnaphunt is seeking a talented Big Data Developer to join their team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.The RequirementsDesign and implement robust data processing solutions using Kafka, PySpark,...

  • Big Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India ZF Group Full time

    Job Title: Big Data DeveloperJob Summary:At ZF Group, we are searching for a skilled Big Data Developer to contribute to the success of our organization. Responsibilities:- Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5 years in Big Data technologies and platforms.- Strong understanding of distributed...

  • Big Data Developer

    2 weeks ago

    Chennai, Tamil Nadu, India Mavinsys Full time

    Data Infrastructure ExpertKey Responsibilities• Design and develop scalable data pipelines using Scala and Spark.• Implement robust and efficient Spark applications for data processing.• Develop RESTful APIs to provide access to processed data for various applications.• Optimize and maintain existing big data infrastructure and pipelines.•...

  • Senior Big Data Engineering Lead

    3 weeks ago

    Chennai, Tamil Nadu, India American Express Full time

    Transform Business with DataAt American Express, we're revolutionizing the way we do business with data. As a Senior Big Data Engineering Lead, you'll play a critical role in shaping our data strategy and delivering innovative solutions that drive growth and customer satisfaction.Key Responsibilities:• Lead a team of engineers in designing and implementing...

  • Big Data Developer

    4 weeks ago

    Chennai, Tamil Nadu, India ZF Group Full time

    Job Title: Big Data DeveloperJob Summary:We are seeking a highly skilled Big Data Developer to contribute to the success of our organization. The ideal candidate will have a strong understanding of distributed computing principles and Big Data ecosystem components, as well as proficiency in programming languages and scripting for data processing and...

  • Big Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India Krishpar Technologies Pvt. Ltd Full time

    We are seeking an experienced Big Data Engineer to join our team at Krishpar Technologies Pvt. Ltd.**Salary:** ₹15,00,000 - ₹25,00,000 per annumAs a Big Data Engineer, you will be responsible for designing and developing scalable data pipelines using Pyspark on Hadoop, Hive, and Kafka.**Required Skills and Qualifications: 4+ years of experience in Big...

  • Senior Big Data Developer

    4 weeks ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time

    Job Title: Senior Big Data DeveloperRole OverviewThe role of a Senior Big Data Developer is crucial in our organization as they are responsible for developing and maintaining efficient, scalable data processing applications using Pyspark.Key Responsibilities* Experience working in big data platforms for implementing projects with high SLA of data...

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Description:As a Big Data Developer at Virtusa, you will be responsible for designing, building, and supporting data applications using Spark, Sqoop, and Hive. You will work with large data sets and high-capacity big data processing platforms, leveraging your expertise in Unix and Shell scripting, as well as your knowledge of the Hadoop ecosystem and its...

  • Big Data Lead

    2 weeks ago

    Chennai, Tamil Nadu, India Hexaware Technologies Full time

    Job Title: Big Data Lead - Senior Data Engineering ProfessionalAbout Us:Hexaware Technologies is a leading global provider of IT services and consulting. We deliver innovative solutions to our clients across various industries.Job Summary:We are seeking an experienced Big Data Lead to join our team in London. The successful candidate will be responsible for...

  • Big Data Lead

    3 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    About the Role:As a Senior Big Data Specialist at Virtusa, you will be responsible for designing, building, and supporting Data Applications using Spark, Sqoop, and Hive. You will work with large data sets and high-capacity big data processing platforms, leveraging your expertise in Unix and Shell scripting, Hadoop ecosystem, and its components.Key...

  • Cloud Architect Lead for Big Data Migration

    4 weeks ago

    Chennai, Tamil Nadu, India NR Consulting - India Full time

    We are seeking a highly skilled Senior Architect with extensive experience in leading large-scale Big Data migration projects to Google Cloud Platform (GCP). Your expertise in GCP technologies, Big Data technologies, Python scripting, and database migration will be crucial in ensuring the successful transition from Hadoop to BigQuery.Roles &...

  • Big Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Hirexa Solutions Full time

    About the Role:We're seeking a seasoned Big Data Developer with a strong background in Hadoop, Spark, Scala, and data pipeline development. As a key member of our team, you'll design and implement efficient data pipelines, develop APIs, and maintain large-scale data infrastructure to support data-driven decision-making.Responsibilities:1. Data Pipeline...