Senior Data Engineer Position

2 weeks ago

Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

Job Overview:

We are seeking an expert in data engineering to work on our team in a state-of-the-art, on-premise environment. The ideal candidate will have hands-on experience with ETL tools, such as Apache Nifi and Talend, along with a solid understanding of data processing technologies like Python scripting and relational databases. Additionally, you should be proficient in working with Spark, PySpark, and other relevant data technologies. While knowledge of cloud tools is beneficial, this position primarily focuses on on-premise data infrastructure.

This is an excellent opportunity to work on exciting projects that require developing scalable data pipelines, real-time data streaming, and optimizing data processing tasks using Python. As a key member of the team, you will be responsible for designing, developing, and optimizing ETL pipelines using open-source or cloud ETL tools. You will also write Python scripts to automate data extraction, transformation, and loading (ETL) processes.

Key Responsibilities:

  • Design and Develop ETL Pipelines: Use ETL tools like Apache Nifi, Talend, and Airflow to design and develop efficient ETL pipelines.
  • Python Scripting for Data Engineering: Write Python scripts to automate data extraction, transformation, and loading (ETL) processes. Ensure that the code is optimized for performance and scalability.
  • Big Data Processing: Work with Apache Spark and PySpark to process large datasets in a distributed computing environment. Optimize Spark jobs for performance and resource efficiency.
  • Job Orchestration: Use Apache Airflow or similar tools to schedule, monitor, and automate data pipeline workflows.
  • Data Streaming: Design and implement real-time data streaming solutions using technologies like Apache Kafka or AWS Kinesis for high-throughput, low-latency data processing.
  • File Formats & Table Formats: Work with open-source table formats like Apache Parquet, Apache Avro, or Delta Lake, and other structured/unstructured data formats for efficient data storage and access.
  • Database Management: Work with relational databases like PostgreSQL, MySQL, and SQL Server for data storage, management, and optimization. Understand database concepts like indexing, normalization, and query optimization.
  • SQL Expertise: Write and optimize complex SQL queries for data extraction, transformations, and aggregation across large datasets. Ensure queries are efficient and scalable.
,

  • Senior Data Engineering Position

    2 weeks ago

    Pune, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering LeadThis is a senior-level data engineering position that requires 4-8 years of experience in designing, building, and optimizing scalable data pipelines on cloud platforms.The ideal candidate will have strong programming skills in Python and/or Scala, hands-on experience with Apache Spark for big data processing, and proficiency with cloud...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionAbout the Role:We are seeking an experienced Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, implementing and maintaining large-scale data warehouses that support our analytics and intelligence platform.Design and develop advanced SQL queries, stored...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Senior Software Engineer

    6 days ago

    Pune, Maharashtra, India Data Axle Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging...

  • Senior QE Engineer

    3 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India SpeedMart Full time

    Company Profile :Our client is a global IT services company that helps businesses with digital transformation with offices in India and the United States.It helps businesses with digital transformation, provide IT collaborations and uses technology, innovation, and enterprise to have a positive impact on the world of business.With expertise is in the fields...

  • Advanced Data Engineer Position

    2 weeks ago

    Pune, Maharashtra, India beBeeDataEngineering Full time ₹ 25,00,000 - ₹ 30,00,000

    Job SummaryAs a Senior Data Engineering professional, you will be responsible for designing and implementing cutting-edge data engineering solutions using various technologies.About the RoleThis is an excellent opportunity to join a dynamic team of innovative thinkers who are committed to disrupting the industry with technology and sustainable business...

  • Senior Data Engineer Position

    1 week ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Title: Data Engineering LeaderWe are seeking an experienced Data Engineering Leader to drive the design and implementation of scalable, secure, and high-performance data pipelines.The ideal candidate will have a strong background in data engineering with expertise in Snowflake, SQL Server, and modern data architecture principles including Medallion...

  • Snowflake Data Engineer Lead Position

    2 weeks ago

    Pune, Maharashtra, India beBeeSnowflake Full time ₹ 18,42,500 - ₹ 23,17,500

    Senior Data Engineer - Snowflake SpecialistWe are seeking an experienced Senior Data Engineer to lead our Snowflake team. The successful candidate will be responsible for designing and developing scalable data pipelines, data models, and transformations using Snowflake.Main Responsibilities:Design, develop, and maintain complex Snowflake databases and data...

  • Lead Data Engineering Position

    2 weeks ago

    Pune, Maharashtra, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Lead Data Engineering PositionWe are seeking a highly skilled and experienced Lead Data Engineer to join our team.The successful candidate will design, develop, and maintain robust data workflows using SSIS, Azure Data Factory, and Databricks.A strong understanding of complex SQL Server queries for data transformation, analysis, and reporting is required.Key...