Big Data Specialist

2 weeks ago

Bengaluru Bangalore Gurgaon Gurugram Chennai, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878
Job Title:
Data Engineer

Key responsibilities include:

  • Work closely with Product Owners and AWS Professional Service Architects to understand requirements, formulate solutions, and implement them.
  • Implement scalable data transformation pipelines as per design - Implement Data model and Data Architecture as per laid out design.
  • Evaluate new capabilities of AWS analytics services, develop prototypes, and assist in drawing POVs, participate in design discussions

Requirements:

  • Minimum 3 years experience implementing transformation and loading of data from a wide variety of traditional and non-traditional sources such as structured, unstructured, and semi structured using SQL, NoSQL and data pipelines for real-time, streaming, batch and on-demand workloads
  • At least 2 years implementing solutions using AWS services such as Lambda, AWS Athena and Glue AWS S3, Redshift, Kinesis, Lambda, Apache Spark,
  • Experience working with data warehousing data lakes or Lakehouse concepts on AWS
  • Experience implementing batch processing using AWS Glue/Lake formation, & Data Pipeline
  • Experience in EMR/MSK
  • Experience or Exposure to AWS Dynamo DB will be a plus
  • Develop object-oriented code using Python, besides PySpark, SQL and one other languages (Java or Scala would be preferred)
  • Experience on Streaming technologies both OnPrem/Cloud such as consuming and producing from Kafka, Kinesis
  • Experience building pipelines and orchestration of workflows in an enterprise environment using Apache Airflow/Control M
  • Experience implementing Redshift on AWS or any one of Databricks on AWS, or Snowflake on AWS
  • Good understanding of Dimensional Data Modelling will be a plus.
  • Ability to multi-task and prioritize deadlines as needed to deliver results
  • Ability to work independently or as part of a team
  • Excellent verbal and written communication skills with great attention to detail and accuracy
  • Experience working in an Agile/Scrum environment

  • Big Data Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Big Data SpecialistExperience:2+ years of experience in big data development.The ideal candidate will have expertise in:Big Data DevelopmentAdvanced AnalyticsCloud ComputingProgramming Skills (Python, Java)As a big data specialist, you will be responsible for designing and developing large-scale data processing systems. Your focus will be on...

  • Big Data Cloud Architect Specialist

    2 weeks ago

    Bengaluru / Bangalore, Gurgaon / Gurugram, India beBeeDatabricks Full time ₹ 15,000 - ₹ 28,00,000

    Job Title:Azure Databricks Platform ExpertAbout the Role:The Impact you will have :Platform Administration: Manage day-to-day administration tasks such as configuring clusters, monitoring platform health, troubleshooting issues, and performing routine maintenance and upgrades.Evaluate new features and enhancements introduced by Azure Databricks from a...

  • Big Data Analytics Specialist

    1 week ago

    Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana, India beBeeData Full time ₹ 1,50,000 - ₹ 28,00,000

    Job Title: Big Data Analytics Specialist\Hadoop Spark and Hive are at the core of our big data strategy. As a Big Data Analytics Specialist, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Hadoop and Spark technologies.\Key Responsibilities:\\Design and develop ETL processes using Hadoop and Spark...

  • Big Data Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data SpecialistWe are seeking a highly skilled professional to develop and maintain large-scale data systems.Key Responsibilities:Design, implement, and deploy data processing pipelines.Develop and maintain ETL processes for big data technologies.Collaborate with cross-functional teams to ensure data quality and integrity.Ensure data security and...

  • Big Data Specialist

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataSpecialist Full time US$ 1,80,000 - US$ 2,00,000

    Job Title: Big Data SpecialistWe are seeking a talented professional to join our team as a Big Data Engineer.About the Role:The ideal candidate will possess strong programming skills in Python and PySpark, along with a solid understanding of HDFS, Hive, and Spark architecture.Key Responsibilities:Building scalable ETL workflows and handling large datasets is...

  • Big Data Quality Assurance Specialist

    6 days ago

    Bengaluru / Bangalore, Gurgaon / Gurugram, Indore, India beBeeDataEngineer Full time ₹ 5,00,000 - ₹ 15,00,000

    We are seeking a skilled professional to fill the role of Data / Big Data Testing Engineer. This position requires strong expertise in data testing and validation, as well as experience with big data tools and technologies.The ideal candidate will have a solid understanding of software development life cycles (SDLC) and system testing life cycles (STLC), as...

  • Big Data Infrastructure Specialist

    2 weeks ago

    Chennai, Pune, India beBeeDataInfrastructure Full time ₹ 1,04,000 - ₹ 1,30,878

    Big Data Infrastructure SpecialistOur company is looking for a skilled Big Data Infrastructure Specialist to join our team. The ideal candidate will be responsible for liaising with coworkers and clients to elucidate the requirements for each task, conceptualizing and generating infrastructure that allows big data to be accessed and analyzed, reformulating...

  • Big Data Specialist

    6 days ago

    Bangalore, India Brillio Full time

    Role Overview We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance...

  • Big Data Operations Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeCloud Full time US$ 90,000 - US$ 1,20,000

    Job Title: Big Data Operations SpecialistKey Responsibilities:Design and implement large-scale data pipelines to process and analyze big data.Collaborate with cross-functional teams to develop and maintain data infrastructure, ensuring high availability and performance.Required Skills and Qualifications:Good experience in working with NoSQL systems, such as...

  • Big Data Specialist

    5 days ago

    bangalore, India Brillio Full time

    Role OverviewWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of...