Lead Big Data Engineer

3 weeks ago

Anywhere in IndiaMultiple Locations, IN Grownex HR Solution Full time

Role : Lead Big Data Engineer

Work Mode : Remote(5 Days working)

The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate would be joining an existing team of outstanding data engineers in the US and help us expand the data engineering team and work on different products and on different layers of the infrastructure.

Job Profile - We is looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company. Applicants must have a passion for engineering with accuracy and efficiency, be highly motivated and organized, able to work as part of a team, and also possess the ability to work independently with minimal supervision.

To be successful in this role, you should possess

- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.

- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.

- Design and implement systems to cleanse, process, and analyze large data sets using distributed processing tools like Akka and Spark.

- Understanding and critically reviewing existing data pipelines, and coming up with ideas in collaboration with Technical Leaders and Architects to improve upon current bottlenecks

- Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.

- 7+ years of experience in developing highly scalable Big Data pipelines.

- In-depth understanding of the Big Data ecosystem including processing frameworks like Spark, Akka, Storm, and Hadoop, and the file types they deal with.

- Experience with ETL and Data pipeline tools like Apache NiFi, Airflow etc.

- Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design Patterns when required.

- Experience with Git and build tools like Gradle/Maven/SBT.

- Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization.

- Have elegant, readable, maintainable and extensible code style.

You are someone who would easily be able to-

- Work closely with the US and India engineering teams to help build the Java/Scala based data pipelines

- Lead the India engineering team in technical excellence and ownership of critical modules; own the development of new modules and features

- Troubleshoot live production server issues.

- Handle client coordination and be able to work as a part of a team, be able to contribute independently and drive the team to exceptional contributions with minimal team supervision

- Follow Agile methodology, JIRA for work planning, issue management/tracking

Additional Project/Soft Skills :

- Should be able to work independently with India & US based team members.

- Strong verbal and written communication with ability to articulate problems and solutions over phone and emails.

- Strong sense of urgency, with a passion for accuracy and timeliness.

- Ability to work calmly in high pressure situations and manage multiple projects/tasks.

- Ability to work independently and possess superior skills in issue resolution.

- Should have the passion to learn and implement, analyse and troubleshoot issues

(ref:hirist.tech)

  • Lead Big Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Grownex HR Solution Full time

    Role : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Big Data Architect

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Architect

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Quality Assurance Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Quality Assurance Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full time

    We are hiring for Big Data AI/ML Engineer - Remote , India Position : Big Data AI/ML EngineerExperience : 6 to 9 YrsLocation : Remote We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology...

  • Big Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full time

    We are hiring for Big Data AI/ML Engineer - Remote , India Position : Big Data AI/ML EngineerExperience : 6 to 9 YrsLocation : Remote We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology...

  • Big Data Architect

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Big Data Architect

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Lead Data Scientist

    7 days ago

    Anywhere in India/Multiple Locations, IN RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a passionate and experienced Lead Data Scientist to join our dynamic team and play a key role in shaping the future of data science within our organization. In this leadership role, you will leverage your expertise to guide complex data projects, manage a team of data scientists, and unlock valuable insights that drive...

  • GCP Data Architect

    2 months ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Role : GCP Data Architect Description : As a GCP Data Architect, you will be responsible for architecting enterprise data solutions for migration, transformation or modernization on Google Cloud Platform (GCP) integrating native GCP services and other 3rd party data solutions. You will collaborate with cloud partners and our cross-functional teams to enable...

  • GCP Data Architect

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Role : GCP Data Architect Description : As a GCP Data Architect, you will be responsible for architecting enterprise data solutions for migration, transformation or modernization on Google Cloud Platform (GCP) integrating native GCP services and other 3rd party data solutions. You will collaborate with cloud partners and our cross-functional teams to enable...

  • Big Data Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data/Databricks Engineer

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN WEN Full time

    Role : Big Data + DatabricksResponsibilities :- Design, develop, and implement scalable Big Data solutions using the Hadoop ecosystem.- Work closely with data scientists and analysts to understand their requirements and translate them into technical solutions.- Develop and optimize Spark and Hive scripts to process, transform, and analyze large datasets.-...

  • Aptus Data Labs

    1 month ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Hadoop Administrator

    2 months ago

    Metros/Anywhere in India/Multiple Locations, IN Dotflick Solutions Full time

    Designation : Hadoop AdministratorOpen for Pan India(Any Location)Job Description :- Development and deployment of data applications- Design Implementation of infrastructure tooling and work on horizontal frameworks and libraries- Creation of data ingestion pipelines between legacy data warehouses and the big data stack- Automation of application back-end...

  • Big Data Architect

    2 months ago

    Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full time

    Job description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...

  • Big Data Architect

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full time

    Job description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...