Big Data Solutions Architect

5 days ago

Kannur, Kerala, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000
Unlock Your Data Potential

We are seeking a highly skilled and innovative Data Engineer to join our team. This is an exciting opportunity to design, develop, and implement scalable data pipelines using Azure Databricks (PySpark/Scala).

Key Responsibilities:
  • Design and Implement Scalable Data Pipelines: Develop and deploy efficient data processing solutions that meet the needs of our business.
  • Develop and Deploy Azure Function Apps: Create event-driven data processing and automation applications that drive business value.
  • Integrate Databricks with Azure Services: Seamlessly integrate Databricks with other Azure services like Data Lake Storage, Synapse, Event Hubs, and Key Vault.
  • Optimize Spark Jobs: Ensure optimal performance and cost-efficiency of Spark jobs.
  • Implement CI/CD Pipelines: Establish and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines for Databricks notebooks and Function Apps.
  • Monitor and Troubleshoot Production Workloads: Proactively identify and resolve issues to ensure high availability and reliability of production workloads.

Candidates will collaborate with data scientists, analysts, and business stakeholders to deliver impactful data solutions.

About the Role

This role requires strong technical expertise in data engineering, excellent problem-solving skills, and the ability to work collaboratively in a fast-paced environment.

Requirements

To be successful in this role, you will need:

  • Expertise in Data Engineering: Proficiency in designing, developing, and deploying data pipelines and architectures.
  • Experience with Azure Services: Familiarity with Azure Databricks, Data Lake Storage, Synapse, Event Hubs, and Key Vault.
  • Strong Programming Skills: Proficiency in languages such as Python, Java, or C#.
  • Problem-Solving Skills: Ability to analyze complex problems and develop effective solutions.
  • Collaboration and Communication: Excellent communication and collaboration skills to work effectively with cross-functional teams.
What We Offer

We offer a competitive salary, comprehensive benefits package, and opportunities for career growth and professional development.

  • Big Data Architect

    1 week ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Opportunity: Big Data ArchitectThe Senior Data Engineer position involves designing, building, and maintaining large-scale data architectures. This includes data lakes, lakehouses, and warehouses.Key Responsibilities:Create scalable real-time and batch processing workflows using PySpark and Apache Spark.Cleanse, transform, deduplicate, normalize, encode,...

  • Experienced Big Data Solutions Architect

    2 weeks ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,20,00,000

    Our client seeks a highly skilled Senior Data Engineer to join their team. In this role, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.You will collaborate closely with data scientists, analysts, and other engineers to ensure the availability,...

  • Big Data Engineer

    3 weeks ago

    Kannur, Kerala, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark)Location: Pune/Nagpur (WFO)Experience: 8 - 12 YearsEmployment Type: Full-timeJob OverviewWe are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Architect

    6 days ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OpportunityLocation: Hyderabad & NoidaWe are seeking a seasoned Data Engineer with 5-8 years of experience to join our team. The ideal candidate will have hands-on experience in Spark, Scala, Big Data (Hadoop), and cloud platforms like AWS/Azure/GCP.Key ResponsibilitiesDesign and develop scalable data processing pipelines using Apache Spark, Kafka, and...

  • Big Data Specialized Solutions Expert

    1 week ago

    Kannur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a seasoned Data Specialist to lead our Big Data endeavors.Our ideal candidate will have extensive experience in handling large-scale data platforms and be proficient in leveraging tools such as PySpark, Hive, and Spark. Additionally, they should possess knowledge of HBase and DQ tool for efficient data ingestion and processing.The successful...

  • Big Data Expert

    1 week ago

    Kannur, Kerala, India beBeeBigData Full time ₹ 18,00,000 - ₹ 30,00,000

    We're seeking a seasoned Data Scientist to spearhead our Big Data initiatives. As a key member of our team, you will have the opportunity to work with cutting-edge technologies and drive business growth through data-driven insights.About the Role:Design, develop, and implement scalable Big Data solutions using PySpark and Hive.Collaborate with...

  • Senior Data Solutions Architect

    4 days ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Job OpportunityWe are seeking a highly skilled Data Engineer to design and build high-availability data and analytics platforms.The ideal candidate will have a strong background in cloud-based solutions, particularly AWS or similar platforms.Develop and maintain complex data pipelines using batch and/or real-time event-based processing techniques.Create ETL...

  • Chief Data Architect

    6 days ago

    Kannur, Kerala, India beBeeBigDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job DescriptionAs a Senior Big Data Engineer, you will be responsible for designing and implementing large-scale data engineering solutions.Our team is looking for an experienced professional to lead our data engineering efforts and work collaboratively with architects, product managers, and operations teams.

  • Data Architect

    1 week ago

    Kannur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Architect RoleEffective data processing is a critical aspect of any organization. As such, we seek an experienced professional to fill the role of Big Data Architect.Key Responsibilities:Design and implement scalable data architectures using Apache Spark and related tools.Configure and maintain data pipelines for seamless data flow.Develop and...

  • Advanced Big Data Architect Role

    2 days ago

    Kannur, Kerala, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Lead Big Data Engineer">We are seeking a talented technical leader to deliver roadmap features of Enterprise TruRisk Platform. This is a fantastic opportunity to be an integral part of a team building Qualys next generation platform using Big Data & Micro-Services based technology.">Key Responsibilities">Be the thought leader in data platform and pipeline...