Lead Big Data Specialist

10 hours ago

Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 1,76,40,000 - ₹ 2,21,80,000
Senior Data Engineer

We are seeking a Senior Data Engineer to join our team. The ideal candidate will have extensive experience in designing, implementing, and maintaining large-scale data processing systems using big data technologies.

Job Description
  • Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements.
  • Develop spark jobs using Scala for new project requirements.
  • Enhance existing spark jobs for any ongoing product enhancement.
  • Performance tuning of spark jobs, stress testing etc.
  • Create new data pipelines for developed/enhanced spark jobs using AWS Lambda or Apache Airflow
  • Responsible for database design process, logical design, physical design, star schema, snowflake schema etc.
  • Analyze data processing, integration, modelling and reporting requirements & Define data loading strategies considering volume, data types, frequency and analytics specifications.
  • Ensure optimal balance between cost and performance.
Required Skills and Qualifications
  • Overall, 7-9 years of IT experience.
  • 5+ years of AWS related project.
  • Good to have Associate Level and Professional Level AWS Certification.
  • In-depth knowledge of following AWS Services required: S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions.
  • Cloud Databases (Must have) – AWS Aurora, Singlestore, RedShift, Snowflake.
  • Big Data (Must have) - Hadoop, Hive, Spark, YARN.
  • Programming Language (Must have) – Scala, Python, Shell Scripts, PySpark.
  • Operating System (Must have) - Any flavor of Linux, Windows.
  • Must have very strong SQL Skills.
  • Orchestration Tools (Must have): Apache Airflow.
Benefits
  • Able to communicate persuasively through speaking, writing, and client presentations.
  • Able to consult, write, and present persuasively.
  • Prior experience of working in a large media company would be an added advantage.
Others
  • Demonstrated continued learning through one or more technical certifications or related methods.
  • Bachelor's degree in computer science, engineering, or related field (Master's degree is a plus).

  • Big Data Integration Specialist

    4 days ago

    Thāne, Maharashtra, India beBeeDataIntegration Full time ₹ 20,00,000 - ₹ 30,00,000

    Big Data Integration SpecialistWe are seeking a seasoned professional to join our team in the field of big data integration. As a key member, you will be responsible for validating data transformations, ingestion, and processing logic in Databricks tables across various layers.Key Responsibilities:Design and execute test cases to validate Databricks...

  • Big Data Specialist

    6 days ago

    Thāne, Maharashtra, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Databricks Data Engineer OpportunityKey Responsibilities:Develop and maintain scalable data pipelines and ETL processes on Databricks.Lead the architecture and implementation of high-performance data solutions.Collaborate with stakeholders to deliver actionable data insights.Optimize data workflows for performance, cost-efficiency, and reliability.Integrate...

  • Big Data Professional

    4 days ago

    Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are looking for a skilled Big Data professional to join our team as a Big Data Engineer"> Big Data Engineer ", we require a candidate with strong skills in data engineering and analytics. The ideal candidate will have hands-on experience with Big Data technologies, including data ingestion, processing, and storage.">Key Responsibilities:Data Ingestion:...

  • Big Data Leader

    3 hours ago

    Thāne, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    ">Senior Data Architect">This role is responsible for leading data engineering teams globally and collaborating with cross-functional teams to drive business outcomes. ">Lead large-scale data engineering projects using distributed systems like Apache Spark, HDFS, and Kubernetes.Design and implement scalable data pipelines for big data processing using Scala,...

  • Big Data Engineering Specialist

    6 days ago

    Thāne, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 16,00,000

    Job DescriptionWe are seeking a skilled Big Data Engineer to join our team. As a key member of our Professional Services, you will play a vital role in designing, building, and maintaining high-performance data pipelines that process massive amounts of data.Key Responsibilities:Design and implement scalable batch processing systems using Python and big data...

  • Senior Big Data Specialist

    9 hours ago

    Thāne, Maharashtra, India beBeeData Full time US$ 90,000 - US$ 1,60,000

    Data Engineering RoleWe are seeking a skilled Data Engineer to join our team.About the Role:This is an exciting opportunity for a seasoned professional with experience in designing and executing large-scale distributed data systems.The ideal candidate will have expertise in big data technologies, including Apache Hadoop, relational databases (e.g., MS SQL...

  • Big Data Developer

    4 weeks ago

    Thāne, Maharashtra, India Tata Consultancy Services Full time

    Role :Data Engineering/Big Data Development Location: HyderabadExperience: 4 to 10 YearsMust Have skills :Bigdata Hadoop Platform -Cloudera

  • Big Data Expert Wanted

    5 days ago

    Thāne, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Professional OpportunityKey Responsibilities:Greetings from a top IT services company.PySparkHiveTo succeed in this role, you should have experience with Big Data platforms and data ingestion from disparate sources. You will work closely with cross-functional teams to drive business outcomes.We offer competitive compensation and benefits packages to...

  • Big Data Leader – Cloud Architect

    7 days ago

    Thāne, Maharashtra, India beBeeCloud Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering LeadAbout the RoleWe are seeking an experienced and strategic data engineering leader with expertise in Big Data platforms, Microsoft Azure, and Databricks. The ideal candidate will lead a team of data engineers to design and deliver scalable data pipelines and analytics platforms.This role is perfect for individuals who enjoy...

  • Principal Big Data Specialist

    9 hours ago

    Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a skilled and experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data processing systems.The ideal candidate will have strong knowledge of data engineering concepts, including data pipelines, distributed systems, and big data...