Big Data Processing Lead

2 days ago

Aligarh, Uttar Pradesh, India beBeeCloudDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Job Title: Data Engineer - Cloud Analytics

About this role : Our team seeks a highly skilled Data Engineer to lead the development and implementation of big data processing, cloud-based analytics, and advanced visualisation technologies.

The ideal candidate will have hands-on experience with data engineering, cloud analytics, or BI development, strong SQL skills, and proficiency in distributed data processing frameworks. Additionally, they should have experience working in Microsoft Azure cloud environments and be proficient in tools like Apache Airflow, Apache Spark, and Databricks.

Key Responsibilities:

  • Design and develop scalable ETL/ELT pipelines using Apache Airflow, Apache Spark, Azure Data Factory, and Databricks.
  • Integrate and manage structured and unstructured data from diverse sources, including Azure Cosmos DB, Azure Data Lake, and Azure Synapse Analytics.
  • Implement IoT data ingestion, processing, and analytics solutions using Azure IoT services.
  • Develop and deploy Python scripts for automation, data transformation, and analytics workflows.
  • Create dashboards, visualizations, and reports to support data-driven decision-making (e.g., Power BI, immersive AR/VR/MR analytics).
  • Collaborate with cross-functional teams to gather requirements, optimize pipelines, and ensure high data quality.
  • Ensure solutions follow security, compliance, and data governance best practices.

Mandatory Skills:

  • Apache Airflow
  • Apache Spark
  • Python
  • Azure Cosmos DB
  • Azure Data Factory
  • Azure IoT
  • Azure Synapse Analytics
  • Azure Data Lake
  • Data bricks
  • AR/VR/MR

Required Experience & Qualifications:

  • 5+ years of experience in data engineering, cloud analytics, or BI development.
  • Strong SQL skills and experience with data modeling.
  • Proficiency in distributed data processing frameworks.
  • Experience working in Microsoft Azure cloud environments.
  • Strong problem-solving skills and ability to work with large-scale datasets.

Preferred Skills:

  • Experience with AI/ML integrations into analytics pipelines.
  • Familiarity with real-time data streaming solutions (Kafka, Event Hubs).
  • Exposure to immersive analytics using AR/VR/MR.

Benefits:

We offer a challenging role in a dynamic environment with opportunities for growth and professional development, a competitive salary and benefits package, and opportunities for career advancement and professional growth.

About us:

We are a leading provider of digital transformation, cloud computing, data analytics, enterprise application integration, infrastructure management, and application development services.

  • Big Data Expert

    2 hours ago

    Aligarh, Uttar Pradesh, India beBeeDataAdministrator Full time ₹ 1,56,00,000 - ₹ 2,02,50,000

    Big Data Administrator JobWe are looking for a skilled professional to manage and maintain our big data platform. The ideal candidate will have experience with Hadoop, Hive, and other big data tools.Administer and monitor Big Data Platform components including Hadoop Hive HBase BigSQL etc.Support Technical and Application team requests for data copies across...

  • Big Data Expert

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeBigData Full time ₹ 15,00,000 - ₹ 25,00,000

    Unlock the Power of Big DataAs a seasoned data specialist, you will be responsible for designing and developing scalable big data architectures using PySpark and Hive.Key Responsibilities:Designing and developing big data architectures to support business growth.Implementing data ingestion from diverse sources onto big data platforms.Collaborating with...

  • Big Data Architect

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job TitleAs a skilled data professional, you will be responsible for designing and implementing large-scale data processing systems using Big Data technologies.">Key ResponsibilitiesDesign and develop PySpark applications using Spark Dataframes in Python.Work on optimizing Spark jobs that process huge volumes of data.Utilize version control tools like Git to...

  • Big Data Engineering Lead

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 16,20,000 - ₹ 24,90,000

    Job Title: Data Engineering LeaderAbout the Role:We seek a strategic and experienced leader to oversee data engineering teams in designing and delivering scalable data pipelines and analytics platforms, ensuring high performance, reliability, and security in cloud-native environments.Key Responsibilities:Lead and mentor a team of talented data engineers...

  • Big Data Solutions Developer

    7 days ago

    Aligarh, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    The objective of this role is to design, develop and implement big data solutions.Key ResponsibilitiesCollaborate with cross-functional teams to create seamless integrations of big data architectures.Ensure high-quality data ingestion from diverse sources onto the big data platform.RequirementsMust-Have: Big Data Architectures, Agile Scrum...

  • Big Data Engineer

    1 week ago

    Aligarh, Uttar Pradesh, India Tata Consultancy Services Full time

    Greetings from TCSTCS is hiring for Big Data Location: - Chennai/Mumbai/PuneDesired Experience Range: 6 to 12 yearsMust-Have • PySpark • HiveGood-to-Have• Spark • HBase • DQ tool • Agile Scrum experience • Exposure in data ingestion from disparate sources onto Big Data platformThanksAnshika

  • Big Data Expertise Wanted

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeDataPlatform Full time US$ 1,50,000 - US$ 2,00,000

    Job Title: Data Platform ArchitectSigmoid is seeking a seasoned professional to lead the development of a high-performance data platform.The ideal candidate will have extensive experience in designing, building, and scaling big data systems. Key responsibilities include:Aligning business objectives with technical requirementsCollaborating with...

  • Big Data Architect

    4 days ago

    Aligarh, Uttar Pradesh, India beBeeDataArchitecture Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewData architects at leading organizations design and implement large-scale distributed data systems.Responsibilities:Designing complex distributed architectures for high-performance applications.Collaborating with cross-functional teams to drive innovation and efficiency.Developing and maintaining scalable solutions that meet business...

  • Senior Big Data Engineer

    2 hours ago

    Aligarh, Uttar Pradesh, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    About the RoleWe are seeking a skilled Senior Big Data Engineer to lead the development and maintenance of our Enterprise TruRisk Platform. This platform helps customers measure, communicate, and eliminate cyber risks.The ideal candidate will have expertise in designing, developing, and supporting highly scalable, distributed SaaS-based security risk...

  • Big Data Expert

    2 days ago

    Aligarh, Uttar Pradesh, India beBeedataAutomation Full time US$ 9,62,177 - US$ 12,49,539

    Job Title: Big Data Tester As a seasoned data automation expert, you will have the opportunity to work with large datasets, comprehend stored procedures, and independently craft data-driven automation logic. Develop and execute test cases with a focus on Fixed Income trading workflows. Key Responsibilities: