Senior Python Data Engineer

1 day ago

Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Data Engineering Position

We are seeking an experienced Data Engineer to join our team in a dynamic on-premise data engineering environment. The ideal candidate will have expertise in ETL tools, data processing technologies, data orchestration, and relational databases.

This is a great opportunity to work on exciting projects that require developing scalable data pipelines, real-time data streaming, and optimizing data processing tasks using Python programming language.

Key Responsibilities:
  • ETL Development & Optimization: Design, develop, and optimize ETL pipelines using open-source or cloud ETL tools.
  • Python Scripting for Data Engineering: Write Python scripts to automate data extraction, transformation, and loading (ETL) processes.
  • Big Data Processing: Work with Apache Spark and PySpark to process large datasets in a distributed computing environment.
  • Job Orchestration: Use Apache Airflow or other orchestration tools to schedule, monitor, and automate data pipeline workflows.
  • Data Streaming: Design and implement real-time data streaming solutions using technologies like Apache Kafka or AWS Kinesis.
  • File Formats & Table Formats: Work with open-source table formats like Apache Parquet or Delta Lake.
  • Database Management: Work with relational databases like PostgreSQL or MySQL for data storage and management.
  • SQL Expertise: Write and optimize complex SQL queries for data extraction and aggregation.
Required Skills & Qualifications:
  • ETL Tools: Experience working with open-source ETL tools like Apache Nifi or Talend.
  • Python Scripting: Proficiency in Python for automating data processing tasks.
  • Big Data Technologies: Experience with Apache Spark and PySpark.
  • Data Orchestration: Experience using Apache Airflow or similar tools.
  • Data Streaming: Experience with Apache Kafka or AWS Kinesis.
  • Relational Databases: Strong experience with at least one relational database.
  • SQL Expertise: Strong skills in writing and optimizing complex SQL queries.
Nice to Have:
  • BI/Analytics Tools: Familiarity with BI tools like Power BI or Tableau.
  • Data Warehousing: Knowledge of data warehousing principles and optimization techniques.
  • Cloud Technologies: Experience with cloud data platforms like Databricks or Snowflake.
Educational Qualifications:
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Additional Qualities:
  • Excellent problem-solving and troubleshooting skills.
  • Ability to work independently and collaboratively.
  • Strong communication skills.
  • Detail-oriented with a focus on data quality and performance optimization.

  • Airflow Senior Data Engineer Opportunity

    3 days ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 10,00,000 - ₹ 19,00,000

    Senior Data Engineer PositionThe ideal candidate will have a strong background in data engineering, with expertise in Airflow and Python. They will be responsible for designing, developing, and maintaining large-scale data processing pipelines using Snowflake.Key Responsibilities:Design and develop complex data processing pipelines using Airflow and...

  • Python AI Engineer

    24 hours ago

    Jamnagar, Gujarat, India Turing Full time

    Work on Real-World Problems with Global Tech ExpertsJoin a leading U.S.-based technology company as a Python AI Engineer, where you'll tackle real-world challenges and build innovative solutions alongside top global experts. This is a fully remote, contract-based opportunity ideal for developers passionate about Python, data analysis, and AI-driven work.Key...

  • Senior Cloud Data Engineer

    7 days ago

    Jamnagar, Gujarat, India beBeeEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Data Engineer PositionAbout the Role:We are seeking an experienced and skilled Senior Cloud Data Engineer to join our team. The ideal candidate will have a strong background in designing, implementing, and optimizing data pipelines using cloud-based technologies.The successful candidate will be responsible for developing scalable and efficient data...

  • Leading Data Engineer

    3 days ago

    Jamnagar, Gujarat, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,10,000

    As a Senior Data Engineer, you will lead the development of scalable data processing systems. Your expertise in cloud-based big data platforms and proficiency in programming languages like Python, Scala, and Spark will enable you to design and implement robust data pipelines.Key Responsibilities:Collaborate with cross-functional teams to integrate data...

  • Senior Data Engineering Specialist

    6 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Are you looking for a challenging role that involves designing, building, and optimizing large-scale data pipelines? We have an exciting opportunity for a Freelance Data Engineer with expertise in Python, SQL, AWS, and Databricks.About the JobThis freelance position requires a skilled professional who can collaborate with internal teams and clients to ensure...

  • Senior Python Programmer

    6 days ago

    Jamnagar, Gujarat, India beBeeBackEndDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    Expert Back-End Developer WantedWe are looking for a highly skilled back-end developer to join our engineering team and contribute to the creation of dynamic software applications for our clients. In this role, you will be responsible for writing and testing scalable code, developing server-side components, and integrating user-facing elements in...

  • Senior Python Developer

    4 days ago

    Jamnagar, Gujarat, India beBeeDeveloper Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:A seasoned Python programmer sought to craft innovative solutions.Key Responsibilities:Develop automated data pipelines using Google Cloud Platform technologies such as BigQuery, Cloud Storage and Vertex AI.Implement analytical algorithms using Python's data processing libraries like Pandas and NumPy.Build REST APIs using frameworks like FastAPI,...

  • Senior Data Engineer

    6 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 23,04,000 - ₹ 2,59,20,000

    Job DescriptionAs a data engineer, you will play a pivotal role in unlocking the full potential of our organization's data assets.Our ideal candidate will possess a unique blend of technical expertise and business acumen, enabling them to design, develop, and maintain scalable data pipelines that drive business growth and inform strategic decision-making.Key...

  • Data Engineering Expert

    3 days ago

    Jamnagar, Gujarat, India beBeeExpertise Full time ₹ 1,80,00,000 - ₹ 2,28,00,000

    Job DescriptionWe are seeking an experienced Senior Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and deploying large-scale data processing systems.Required Skills and QualificationsProven experience leading data engineering teams, including distributed teams across multiple...

  • Lead Data Engineering Expert

    13 hours ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior ETL Developer Job DescriptionWe are seeking a highly experienced Senior ETL Developer to join our team.The ideal candidate will have a strong background in data engineering and experience working with big data technologies.Main Responsibilities:Collaborate with stakeholders to understand data requirements and translate them into technical...