Data Engineer Position

5 days ago

Ellore, Andhra Pradesh, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,50,00,000
Job Opportunity: Data Processing Expert

We are seeking a skilled professional to join our organization as a data processing expert in an on-premise environment. The ideal candidate will have experience in ETL tools, data processing technologies, and relational databases.

This is an excellent opportunity to work on exciting projects that require developing scalable data pipelines and optimizing data processing tasks using Python.

Key Responsibilities:

  • Design, develop, and optimize ETL pipelines using open-source or cloud ETL tools (e.g., Apache Nifi, Talend, Pentaho, Airflow, AWS Glue).
  • Write Python scripts to automate data extraction, transformation, and loading (ETL) processes.
  • Work with Apache Spark and PySpark to process large datasets in a distributed computing environment.
  • Use Apache Airflow or other orchestration tools to schedule, monitor, and automate data pipeline workflows.
  • Design and implement real-time data streaming solutions using technologies like Apache Kafka or AWS Kinesis.
  • Work with open-source table formats like Apache Parquet, Apache Avro, or Delta Lake for efficient data storage and access.
  • Work with relational databases (e.g., PostgreSQL, MySQL, SQL Server) for data storage, management, and optimization.
  • Write and optimize complex SQL queries for data extraction, transformations, and aggregation across large datasets.

Required Skills & Experience:

  • Experience working with open-source ETL tools such as Apache Nifi, Talend, or Pentaho.
  • Proficiency in Python for automating data processing tasks and working with libraries such as Pandas, Dask, PySpark, etc.
  • Experience with Apache Spark and PySpark for distributed data processing, along with optimization techniques.
  • Experience using Apache Airflow or similar tools for scheduling and automating data pipelines.
  • Experience with Apache Kafka or AWS Kinesis for building and managing real-time data pipelines.
  • Knowledge of Apache Parquet, Apache Avro, Delta Lake, or similar open-source table formats for efficient data storage and retrieval.
  • Strong experience with at least one relational database (e.g., PostgreSQL, MySQL, SQL Server) and a solid understanding of database concepts like indexing, normalization, and query optimization.

Nice to Have:

  • Familiarity with BI tools like Power BI, Tableau, Looker, or similar reporting and data visualization platforms.
  • Knowledge of data warehousing principles, schema design (e.g., star/snowflake), and optimization techniques for large datasets.
  • Experience with cloud data platforms like Databricks, Snowflake, or Azure Synapse is beneficial, though the role is focused on on-prem environments.
  • Familiarity with containerization tools like Docker or Kubernetes for deploying data engineering workloads.

Educational Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent work experience).

Additional Qualities:

  • Excellent problem-solving and troubleshooting skills.
  • Ability to work both independently and in a collaborative environment.
  • Strong communication skills, both written and verbal.
  • Detail-oriented with a focus on data quality and performance optimization.
  • Proactive attitude and the ability to take ownership of projects.

  • Cloud Data Engineer Position

    6 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,70,00,000

    Job Title: Cloud EngineerWe are seeking a skilled Data Engineer to fill a long-term contract position with our organization. The ideal candidate will have expertise in Google Cloud Platform (GCP) and a strong background in data engineering.The successful candidate will be responsible for developing, constructing, testing, and maintaining data acquisition...

  • Data Cloud Engineer Position

    5 days ago

    Ellore, Andhra Pradesh, India beBeeCloud Full time ₹ 15,00,000 - ₹ 20,00,000

    ETL Tester With Cloud ExpertiseWe are seeking an experienced ETL tester to join our team. The ideal candidate will have strong testing skills and experience working with cloud platforms.The successful candidate will have a deep understanding of complex system requirements, enabling them to design effective test cases.They will be proficient in developing and...

  • Freelance Data Engineering Position

    1 week ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Freelance Data Engineer OpportunityA seasoned data professional with expertise in Python and AWS services is required to design, develop, and optimize large-scale data systems.The ideal candidate will have strong coding skills in Python, experience in PySpark, and hands-on knowledge of AWS Glue, Step Functions, and S3.Develop scalable data pipelines using...

  • Data Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 20,10,000

    Data Engineer PositionWe are looking for an experienced data engineer to create and maintain data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform.Key Responsibilities:Design and implement end-to-end pipelines from Oracle Symphony (API extraction) into Databricks Lakehouse.Develop efficient...

  • Senior Data Engineer Position

    1 week ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Big Data DeveloperWe're looking for a skilled Data Scientist to join our team. As a key member of our analytics and business teams, you'll be responsible for developing and maintaining scalable data pipelines to support our growing data needs.Key Responsibilities:Develop and maintain large-scale data systems using Bigdata, Pyspark, Python, Hadoop/HDFS,...

  • Key Senior Data Engineering Position

    3 days ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer RoleWe're looking for a skilled data engineer to join our team. As a senior data engineer, you'll be responsible for designing, developing, and operating data warehouses and data platforms.A bachelor's degree in computer science, computer engineering, or electrical engineering or equivalent technical background and experience5+ years of...

  • Senior Data Engineering Position

    6 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 35,00,000

    Job Title:Data Engineering ProfessionalThis is an exciting opportunity to join our organization as a seasoned Data Engineer, working with a talented team to design and develop cutting-edge data solutions.Key Responsibilities:Lead the analysis of existing data integration frameworks to identify opportunities for improvement and implement new...

  • Data Engineer

    1 week ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Data Engineer - Design and Deliver Scalable Data SolutionsWe are seeking a skilled data engineer to join our team. In this role, you will design, develop, and maintain scalable data pipelines and data warehouse solutions.Key Responsibilities:Design, build, and optimize data pipelines on Google Cloud Platform.Develop and maintain data warehouse solutions to...

  • Senior Data Engineering Professional

    2 days ago

    Ellore, Andhra Pradesh, India beBeeDataPipeline Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Pipeline Engineer PositionWe are seeking a skilled and motivated Data Pipeline Engineer to design, develop, and maintain scalable data pipelines using IBM DataStage, AWS Glue, and Snowflake.

  • Data Engineering Specialist

    4 days ago

    Ellore, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 8,00,000 - ₹ 10,00,000

    Job Title: Data Engineering SpecialistWe are seeking a skilled professional to fill the role of Data Engineering Specialist. This position will involve designing, developing, and maintaining efficient data systems and workflows.Key Responsibilities:Design and implement data pipelines to extract and transform large datasets.Develop Python scripts for data...