Big Data Engineer

2 weeks ago


India Insight Global Full time

Key Responsibilities

  • Build and maintain data pipelines to enable faster, better, data-informed decision-making through customer enterprise business analytics
  • Support ETL processes, fixing errors, repairing data, upgrading designs and code for resiliency
  • Monitor and support data platforms and services: respond to alerts, improve process efficiency, reduce costs, manage data quality and integrity
  • Ensure data integrity within reports and dashboards by reviewing data, identifying and resolving gaps and inconsistencies, and escalating as required to foster a partnered approach to data accuracy for business reporting purposes.
  • Manage existing cloud-based storage solutions such as blob storage, databases, and data-warehouses
  • Measure throughput, reliability and latency of data ingestion and processing for development, testing, and production environments
  • Creation of analytical reports and/or dashboards
  • Create and maintain documentation on data processes, policies, and configurations
  • Provide on-call support for production systems as needed
  • Continuously evaluate current systems and processes against industry standards and best practices

Required Skills

  • Experience working with cloud-native data services and platforms such as Data Factory, Data Bricks, PerView, UnityCatalog, Calibra Data Marketplace, Synapse and Snowflake
  • Strong skillsets within Azure Cloud Data Service Offerings:Kafka, ADF, ADLS Gen2, Databricks, PowerBI, Tableau, Snowflake, data pipeline orchestration
  • Working proficiency building and managing ETL pipelines (data extraction, transformation, and loading),and data storage solutions
  • Experience working with big data technologies such as Hadoop, Apache Hive, Redshift, Kafka, or Apache Spark
  • Strong software development or scripting background with Python and/or C#
  • In-depth understanding of data ingestion architecture, processes and best practices
  • Strong experience working with continuous integration tools such as Jenkins, GitLab, etc.
  • Understanding of data modeling, algorithms, and data transformation techniques
  • Experience managing data integrity and quality for enterprise analytical reporting
  • Ability to understand and adapt to communicated requirements from the client or team leads
  • Experience providing production level support for labs/sandbox services
  • Excellent communication and time management skills


  • Big Data Engineer

    3 weeks ago


    India TekJobs Full time

    This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value.Job Background/context:The technology team is responsible for building and maintaining a multitude of applications including:Real-time...

  • Big Data Engineer

    4 weeks ago


    india TekJobs Full time

    This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value. Job Background/context: The technology team is responsible for building and maintaining a multitude of applications including: Real-time...

  • Big Data Engineer

    3 days ago


    india NeoStats Full time

    Job Description About Neostats Analytics: NeoStats is a new-age, Data & Analytics firm offering contemporary solutions and infinite possibilities. Our mission is to create lasting competitive advantage for our clients by transforming them into world-class, data-driven organizations. Established in 2022 to provide End to End Data & Analytics Services, we...

  • Data Engineer

    2 days ago


    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • AWS Big Data Engineer

    1 month ago


    india Bosch Full time

    Job Description Technical skills ETL development experience with overall 4 to 7 years in AWS , Big data engineering related application Minimum 2 end to end ETL project implementation Proficient in PySpark, Python4. Proficient in Big Data technologies - Hadoop, Spark, Hive, Databricks5. Expert in AWS to develop, deploy and maintain ETL...

  • Data Engineer –

    4 weeks ago


    india HARMAN International Full time

    HARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...

  • Senior Data Engineer

    1 month ago


    india SWAI TECHNOLOGIES PRIVATE LIMITED Full time

    Job Description : We are seeking a highly skilled Senior Data Engineer with 5 - 12 years of experience in building scalable data solutions. The ideal candidate should be proficient in Python programming and SQL, with extensive experience in utilizing various AWS services for data processing and storage. Experience with big data processing environments,...

  • Big Data Engineer

    1 month ago


    india Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...

  • Big Data Architect

    3 weeks ago


    India Bloom Consulting Services Full time

    NA Experience **12 - 16 years** Offered Salary Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end tools). 9+ years of...

  • Big Data Architect

    3 weeks ago


    India Bloom Consulting Services Full time

    Not Disclosed Experience **12 - 16 Years** Offered Salary **Lakh** Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end...

  • Big Data Engineer

    1 month ago


    india ATech Full time

    Designation: BIG DATA ENGINEERJob Description:Your Role and Responsibilities:- Understand a data warehousing solution and able to work independently in such an environment- Responsible in Project development and delivery experience of a few good size projects- Design, build, optimize and support new and existing data models and ETL processes based on our...


  • Bangalore,Anywhere in India,Multiple Locations Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...


  • Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Analyst

    1 month ago


    india Royal Cyber ​​Inc. Full time

    1) 5+ years of experience managing and deploying CDP or HDP clusters.2) Experience configuring and setup of big data environments.3) Experience with Kerberos security and LDAP setup.4) Experience in setting up monitoring and alerting pipelines for big data clusters.

  • Big Data Developer

    3 weeks ago


    india RiskInsight Consulting Pvt Ltd Full time

    We are seeking a skilled Data Engineer with 4-8 years of experience, proficient in Big Data technologies and data processing frameworks. The ideal candidate will have strong expertise in Hadoop ecosystem components such as HDFS, HiveQL, and experience in developing and optimizing data processing workflows using PySpark. Proficiency in SQL for data...


  • india Digihelic Solutions Pvt. Ltd. Full time

    Job Summary :We are seeking a highly skilled and motivated Product Solution Architect with extensive expertise in designing and implementing cutting-edge big-data solutions on the Amazon Web Services (AWS) platform.As a key member of our dynamic team, you will play a pivotal role in driving the success of our data-driven initiatives, leveraging AWS services...


  • Anywhere in India,Multiple Locations Grownex HR Solution Full time

    Role : Lead Big Data Engineer Work Mode : Remote(5 Days working) The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The...


  • Anywhere in India/Multiple Locations, IN Grownex HR Solution Full time

    Role : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Big Data Engineer

    1 month ago


    Bangalore,Anywhere in India,Multiple Locations Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWS Job Description : - Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data Engineer

    4 weeks ago


    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...