Senior Data Engineering Expert

1 week ago

Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 18,00,000

Job Summary

Main Responsibilities
  • Conduct an assessment of the current data integration framework to identify areas for improvement and alignment with new requirements.
  • Design and develop scalable Spark jobs using Scala to meet project needs, ensuring efficient execution and minimal latency.
  • Analyze existing Spark jobs to enhance their performance, functionality, and maintainability, in line with ongoing product enhancement objectives.
  • Optimize Spark job performance through stress testing and issue resolution, guaranteeing optimal results within given constraints.
  • Create data pipelines for developed/enhanced Spark jobs utilizing AWS Lambda or Apache Airflow, ensuring seamless data flow and minimal bottlenecks.
  • Develop database designs, encompassing logical design, physical design, star schema, and snowflake schema, to optimize data storage and retrieval efficiency.
  • Analyze data processing, integration, modeling, and reporting requirements to define data loading strategies based on volume, data types, frequency, and analytics specifications.
  • Maintain a balance between cost and performance, ensuring maximum ROI without compromising data integrity or system reliability.
  • Ensure project documentation adheres to quality guidelines and schedules, facilitating smooth collaboration and knowledge sharing among team members.
  • Collaborate with project managers to deliver projects successfully, providing estimation, scoping, and scheduling assistance to ensure timely completion.
Technical Requirements
  • A minimum of 7-9 years of IT experience, with at least 5+ years of AWS-related projects.
  • Good to have Associate Level and Professional Level AWS Certification.
  • In-depth knowledge of AWS Services: S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions.
  • Proficiency in Cloud Databases: AWS Aurora, Singlestore, RedShift, Snowflake.
  • Expertise in Big Data: Hadoop, Hive, Spark, YARN.
  • Familiarity with Programming Languages: Scala, Python, Shell Scripts, PySpark.
  • Operating System: Any flavor of Linux, Windows.
  • Strong SQL skills.
  • Orchestration Tools: Apache Airflow.
  • Ability to develop ETL workflows with complex transformations like SCD, deduplications, aggregations.
  • Understanding of AWS VPC, Subnets, Security Groups, & Route Tables.
  • Capacity to adapt quickly to new AWS Services or Big Data Technologies as needed.
Qualifications
  • Bachelor's degree in computer science, engineering, or related field (Master's degree is a plus).
  • Demonstrated continued learning through certifications or related methods.
  • Minimum 5 years of experience in Cloud-related projects.

We are seeking a skilled and experienced professional to join our team as a Senior Data Engineer. This role requires a strong background in data engineering, cloud computing, and big data technologies. The ideal candidate will have a proven track record of designing and implementing scalable data pipelines, developing efficient Spark jobs, and optimizing database performance.

  • Senior Data Protection Expert

    6 days ago

    Nellore, Andhra Pradesh, India beBeeDataProtection Full time ₹ 18,00,000 - ₹ 21,00,000

    Job Role: Senior Data Protection ExpertAbout the Role:We are seeking a highly skilled Senior Data Protection Expert to join our team.The successful candidate will have extensive experience in encryption technologies, data security, and cybersecurity analysis.You will collaborate with cross-functional teams to identify vulnerabilities and implement controls.A...

  • Senior Data Protection Expert

    6 days ago

    Nellore, Andhra Pradesh, India beBeeDataProtection Full time ₹ 20,00,000 - ₹ 40,00,000

    Job DescriptionWe are seeking a Senior Data Protection Expert to join our team. As a key member of our organization, you will be responsible for ensuring the highest standards of data protection and privacy across our operations.The ideal candidate will have extensive knowledge of global data protection laws, including U.S. federal and state student data...

  • Data Engineering Expert

    6 days ago

    Nellore, Andhra Pradesh, India beBeeSearchEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Expert Search Solutions EngineerThe ideal candidate will combine expertise in Azure Cognitive Search and data engineering with a passion for delivering intuitive user experiences and business value.Key ResponsibilitiesArchitect and implement search solutions using Azure Cognitive Search and related AI services.Build pipelines to ingest, index, and enrich...

  • Expert Data Pipelines Engineer

    2 weeks ago

    Nellore, Andhra Pradesh, India beBeeDataIntegration Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Integration SpecialistJob Title: Senior Consultant Data EngineeringWe are seeking a highly skilled data integration specialist to join our team.The ideal candidate will be responsible for designing and developing end-to-end data pipelines, optimizing workflows, and collaborating with business and technical stakeholders to deliver scalable and...

  • Senior Cloud Data Engineer

    1 week ago

    Nellore, Andhra Pradesh, India beBeeCloud Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Cloud Data EngineerWe are seeking a seasoned Senior Cloud Data Engineer to spearhead the development of end-to-end data pipelines on cloud infrastructure.Key Responsibilities:Data Pipeline Development: Design, develop, and deploy scalable data pipelines using cloud services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.Data Processing...

  • Senior Data Architect

    1 week ago

    Nellore, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Architect - AI Engineering LeadWe are seeking an experienced Data Architect to lead our data and AI engineering initiatives.About the Role:The ideal candidate will have hands-on experience in designing, implementing, and scaling data platforms and AI systems.Responsibilities include developing end-to-end machine learning models from...

  • Senior Data Visualization Experts

    1 week ago

    Nellore, Andhra Pradesh, India beBeeVisualization Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We are seeking senior-level professionals to support a project focused on creating a dataset of visualizations for LLM training.This role requires not just technical expertise but also the ability to deliver production-level code with a strong emphasis on quality and scalability.Key responsibilities include:Designing and developing high-quality...

  • Expert Data Architect

    1 week ago

    Nellore, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Role:We are looking for a highly skilled professional to fill the role of Senior Data Engineer.The ideal candidate will be responsible for designing, deploying, and maintaining data platforms to drive business growth.The key responsibilities of this role include:Reviewing and analyzing existing data collection systems and recommending improvementsLeading...

  • Expertise in Advanced Data Engineering

    6 days ago

    Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Engineer Job OpportunityWe are seeking a seasoned Senior Data Engineer to join our organization. The ideal candidate will have strong expertise in SQL and a good understanding of data from an analytical perspective.The role requires proficiency in working with JSON structures and REST APIs, as well as the ability to interpret legacy system code....

  • Visionary Data Specialist

    1 week ago

    Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data EngineerAbout Us:We are a leading IT services and consulting company with a strong presence globally. Our focus on digital transformation, technology consulting, and business process outsourcing has established us as a key player in the industry.Key Skills: Azure, Data EngineeringJob Location: PAN IndiaExperience: 8+ YearsEducation Qualification:...