Principal Big Data Developer

3 days ago

Jodhpur, Rajasthan, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,50,00,000
Job Overview:

We are seeking an experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka.

This role involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems using Spark (PySpark) and Python. The candidate will also work extensively with Apache Kafka for streaming solutions and orchestration tools like NiFi / Airflow.

The successful applicant will write efficient queries, perform data analysis using Jupyter Notebooks and SQL, and collaborate with cross-functional teams to design scalable cloud-based data architectures. Additionally, they will ensure the delivery of high-quality code through code reviews, performance tuning, and best practices.

Key Responsibilities:

  • Design and develop large-scale data pipelines and distributed data processing systems.
  • Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.
  • Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.
  • Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.
  • Collaborate with cross-functional teams to design scalable cloud-based data architectures.
  • Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

Mandatory Skills and Qualifications:

  • Scala - Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark (PySpark) and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi / Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
  • Good understanding of Cloud Architecture.

Preferred Skills:

  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive, responsive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.
  • Hands-on experience with CI/CD tools (Git, Jenkins).
  • Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Big Data Platform Leader

    3 days ago

    Jodhpur, Rajasthan, India beBeeTechnicalLead Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Our company is looking for a seasoned leader to spearhead the development of a high-performance big data platform. This innovative solution will enable us to collect, store, model, and analyze massive datasets from diverse sources.To achieve this goal, we require an individual with 9+ years of experience in relevant work and a strong educational background...

  • Senior Big Data Specialist

    7 days ago

    Jodhpur, Rajasthan, India beBeeBigData Full time ₹ 10,00,000 - ₹ 12,00,000

    VicTree Solutions is currently seeking a Senior Big Data Specialist to join our team on a contract basis.About the Role:This position offers the opportunity to work with retail offline and online data sets, utilizing your expertise in Google Cloud Platform and Azure Databricks.You will be responsible for designing and implementing data pipelines, leveraging...

  • Principal Data Architect

    5 hours ago

    Jodhpur, Rajasthan, India Tech Anand Rathi - ARIT Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Title: Principal Data Architect (on Contract)Location:Jodhpur (Hybrid)Department:Data Engineering & AnalyticsCompany:AnandRathi IT Pvt Ltd ServicesAbout Us:AnandRathi is one of India's leading financial services firms, offering a wide range of services in wealth management, capital markets, brokerage, investment banking, and financial planning. As part...

  • Big Data Solutions Architect

    5 days ago

    Jodhpur, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer RoleWe are seeking a highly skilled Data Engineer to join our team.Key responsibilities will include designing, developing, and maintaining large-scale data systems.Candidates should have experience with Big Data technologies, including AWS, Python, and PySpark.The ideal candidate will have a strong background in software development, excellent...

  • Financial Data Platform Developer

    1 week ago

    Jodhpur, Rajasthan, India beBeeDeveloper Full time US$ 2,00,000 - US$ 2,50,000

    Unlock New Horizons in Financial Innovation">">Develop a cutting-edge platform for financial data management, harnessing the power of big data systems and technologies like Apache Spark and Hive.">Collaborate with cross-functional teams to define strategic roadmaps and drive technical excellence, leveraging expertise in software engineering and...

  • Chief Data Architect

    2 days ago

    Jodhpur, Rajasthan, India beBeeDataEngineer Full time US$ 6,00,000 - US$ 10,00,000

    Big Data Engineer Job DescriptionWe are seeking an experienced Big Data Engineer to design and implement scalable big data solutions.Key Responsibilities:Create data pipelines for ingestion from disparate sources onto big data platforms.Evaluate and improve data processing and analytics to ensure high-quality results.Collaborate with cross-functional teams...

  • Big Data Solutions Specialist

    3 days ago

    Jodhpur, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data EngineerWe are seeking a self-driven Data Architect with 4–7 years of experience to build and optimize scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.The role involves working across scrum teams to develop data solutions, ensure data governance with Unity Catalog, and support real-time and batch processing.Strong...

  • Big Data Solutions Architect

    5 days ago

    Jodhpur, Rajasthan, India beBeeAzureEngineer Full time US$ 80,000 - US$ 1,50,000

    Job OverviewWe are seeking an experienced Azure Data Engineer to join our development and support team. You will be responsible for designing, implementing, and maintaining data and analytics solutions using Azure Data Engineering technologies with a strong focus on performance, scalability, and reliability.The ideal candidate will have expertise in...

  • Senior Big Data Engineer

    3 days ago

    Jodhpur, Rajasthan, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Data Engineering PositionWe are seeking an experienced Data Engineer to join our team. As a key member of our data engineering team, you will play a critical role in designing, developing, and maintaining our data pipelines.

  • Data Engineering Leader

    2 days ago

    Jodhpur, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled data engineer to design and implement scalable, high-performance data pipelines. This role involves building and optimizing ETL/ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI/ML, and business intelligence use cases.The ideal candidate will...