Lead Big Data Engineer

4 weeks ago

Anywhere in IndiaMultiple Locations, IN Grownex HR Solution Full time

Role : Lead Big Data Engineer

Work Mode : Remote(5 Days working)

The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate would be joining an existing team of outstanding data engineers in the US and help us expand the data engineering team and work on different products and on different layers of the infrastructure.

Job Profile - We is looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company. Applicants must have a passion for engineering with accuracy and efficiency, be highly motivated and organized, able to work as part of a team, and also possess the ability to work independently with minimal supervision.

To be successful in this role, you should possess

- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.

- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.

- Design and implement systems to cleanse, process, and analyze large data sets using distributed processing tools like Akka and Spark.

- Understanding and critically reviewing existing data pipelines, and coming up with ideas in collaboration with Technical Leaders and Architects to improve upon current bottlenecks

- Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.

- 7+ years of experience in developing highly scalable Big Data pipelines.

- In-depth understanding of the Big Data ecosystem including processing frameworks like Spark, Akka, Storm, and Hadoop, and the file types they deal with.

- Experience with ETL and Data pipeline tools like Apache NiFi, Airflow etc.

- Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design Patterns when required.

- Experience with Git and build tools like Gradle/Maven/SBT.

- Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization.

- Have elegant, readable, maintainable and extensible code style.

You are someone who would easily be able to-

- Work closely with the US and India engineering teams to help build the Java/Scala based data pipelines

- Lead the India engineering team in technical excellence and ownership of critical modules; own the development of new modules and features

- Troubleshoot live production server issues.

- Handle client coordination and be able to work as a part of a team, be able to contribute independently and drive the team to exceptional contributions with minimal team supervision

- Follow Agile methodology, JIRA for work planning, issue management/tracking

Additional Project/Soft Skills :

- Should be able to work independently with India & US based team members.

- Strong verbal and written communication with ability to articulate problems and solutions over phone and emails.

- Strong sense of urgency, with a passion for accuracy and timeliness.

- Ability to work calmly in high pressure situations and manage multiple projects/tasks.

- Ability to work independently and possess superior skills in issue resolution.

- Should have the passion to learn and implement, analyse and troubleshoot issues

(ref:hirist.tech)

  • Big Data Developer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Risk Resources LLP Full time

    Responsibilities :- Lead and mentor a team of Big Data engineers in designing, developing, and deploying big data solutions on GCP- Define the technical vision and roadmap for big data architecture within the organization- Architect and implement scalable, high-performance big data pipelines using technologies like MapReduce, Hive, Spark, and BigQuery-...

  • Big Data Quality Assurance Engineer

    4 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Architect

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Big Data Engineer

    4 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • GCP Data Architect

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Role : GCP Data Architect Description : As a GCP Data Architect, you will be responsible for architecting enterprise data solutions for migration, transformation or modernization on Google Cloud Platform (GCP) integrating native GCP services and other 3rd party data solutions. You will collaborate with cloud partners and our cross-functional teams to enable...

  • Aptus Data Labs

    4 days ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Hadoop Administrator

    4 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN Dotflick Solutions Full time

    Designation : Hadoop AdministratorOpen for Pan India(Any Location)Job Description :- Development and deployment of data applications- Design Implementation of infrastructure tooling and work on horizontal frameworks and libraries- Creation of data ingestion pipelines between legacy data warehouses and the big data stack- Automation of application back-end...

  • Big Data Architect

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full time

    Job description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...

  • Valiance Solutions

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Valiance Analytics Pvt. Ltd. Full time

    About Us :Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Paralleldots Full time

    Job Description :As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and :- Lead the design,...

  • Data Engineer

    4 weeks ago

    Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full time

    Basic understanding of :- Scheduling and workflow management & working experience in either ADF, Informatica, Airflow, or Similar- Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner, or Similar- Logical/Physical model on Big Data sets or modern data warehouse & working experience in ERwin, ER/Studio,...

  • MDM Architect

    4 weeks ago

    Anywhere in India/Multiple Locations, IN TELL Jobs Full time

    Job Description :- Minimum 8+ years of proven experience and Own and aggressively drive forward MDM, Data Governance, Big Data, and Cloud Data Management- Develops data integration processes using the Informatica platform - Creating business solutions which utilize the CDI platform and ensure high quality and scalable solutions- Work with EA (Enterprise...

  • Data Engineer

    2 weeks ago

    Anywhere in India/Multiple Locations, IN AAR Consultancy Full time

    Technical Skills :1. Google Cloud Platform (GCP) : In-depth knowledge and hands-on experience with GCP services such as BigQuery, Dataflow, Dataproc, Pub/Sub, Data Studio, and Cloud Storage.2. Data Engineering : Strong understanding of data engineering principles, including data ingestion, transformation, processing, and storage.3. Big Data Technologies :...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Primary Roles and Responsibilities:- Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack- Ability to provide solutions that are forward-thinking in data engineering and analytics space- Collaborate with DW/BI leads to understand new ETL pipeline development requirements.- Triage issues to find gaps in existing pipelines and fix...

  • MDM Architect

    4 weeks ago

    Anywhere in India/Multiple Locations, IN TELL Jobs Full time

    Job Description :- Minimum 8+ years of proven experience and Own and aggressively drive forward MDM, Data Governance, Big Data, and Cloud Data Management- Develops data integration processes using the Informatica platform (eg- Creating business solutions which utilize the CDI platform and ensure high quality and scalable solutions- Work with EA (Enterprise...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations/Bangalore, IN Mobile Programming LLC Full time

    Job Description :Position : Data EngineerExperience : 6+ yearsWork Mode : Work from OfficeLocation : BangaloreExperience in Nifi or Tibco is mandatory.Mandatory Skills : ETL, DevOps platform, Nifi or TibcoPlease note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data...

  • Havells - Data Engineer - ETL/Hadoop/Tableau

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Havells India Limited Full time

    Must have : - 3-7 years of meaningful experience in data engineering working on building ETL pipelines, big data tech like spark- Very good expertise in algorithms and data structures- Excellent in at least one programming language: Python, Go, R- Hands-on expertise big data technologies such as Apache Spark, Hadoop, message queues(Kafka), Apache Airflow,...

  • Lead/Senior Solution Engineer

    6 days ago

    Anywhere in India/Multiple Locations, IN Recruit Elite Full time

    Job Description :Role & Responsibilities :- Work across the entire landscape that spans across network, compute, storage, data- bases, applications and business domain- Use the Big Data and AI driven features of vuSmartMaps to provide solutions that will enable customers to improve the end user experience for their applications- Create detailed designs,...

  • ETL Developer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Dotflick Solutions Full time

    Job Designation : ETL DeveloperWFH / WFOJob Description :- Work Experience : Overall 3+ years of experience working on Databases, Data Warehouse, Data Integration and BI/Reporting solutions with relevant experience.Role & Responsibilities :- To be a Data expert/Architect in implementing solutions using Databases/ETL/DWH/Data Integration/Reporting tools with...

  • Nutrabay - Data Engineer - Data Warehousing & Integrity

    4 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Nutrabay Full time

    Responsibilities :- Design and implement data pipelines for extracting, transforming, and loading data from various sources to our data warehouses and data lakes.- Collaborate with data scientists and analysts to understand data requirements and ensure data quality and integrity.- Optimize and tune data pipelines for performance and scalability.- Develop and...