Lead Big Data Solution Developer

2 days ago

Gulbarga, Karnataka, India beBeePySparkEngineer Full time ₹ 12,00,000 - ₹ 16,00,000
Senior Data Solutions Engineer

Role Overview:

This is a contract remote role seeking an experienced Senior Data Solutions Engineer with 3-4 years of experience in building, optimizing, and maintaining large-scale data processing solutions using PySpark.

Key Responsibilities:

  • Design, develop, and optimize complex data pipelines using PySpark, Spark, and Python.
  • Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
  • Develop and maintain Hive queries and ensure seamless data transformations.
  • Write and optimize SQL queries for large datasets.
  • Build, monitor, and support real-time streaming solutions using Apache Kafka.
  • Work on Databricks to create, manage, and optimize data workflows.
  • Deploy and manage solutions in Azure cloud services.
  • Debug, troubleshoot, and optimize code for performance and scalability.
  • Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills & Qualifications:

  • 3-4 years of experience in data engineering with a focus on PySpark and Python.
  • Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
  • Experience with Databricks in a production environment.
  • Hands-on experience with Azure cloud services.
  • Experience working with Apache Kafka or other real-time streaming technologies.
  • Strong problem-solving and debugging skills.
  • Excellent written and verbal communication skills.

Nice-to-Have Skills:

  • Exposure to CI/CD pipelines for data solutions.
  • Knowledge of data governance and security best practices.
  • Experience with Agile/Scrum methodology.

  • Big Data Solutions Architect

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineer PositionWe are seeking an experienced Data Engineer to design and implement big data solutions.The ideal candidate will have expertise in distributed computing principles, Apache Spark, and Python programming.Key responsibilities include designing and implementing big data solutions using technologies like Apache Spark, Hadoop, and NoSQL...

  • Senior Big Data Solutions Developer

    4 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Big Data Engineer RoleWe are seeking a highly skilled Data Engineer to join our team and contribute to the development of innovative big data engineering solutions. The ideal candidate will have expertise in cloud technologies, data processing architectures, and data pipelines.Key Responsibilities:Design, build, and maintain robust data pipelines for...

  • Senior Big Data Architect

    15 hours ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    As a senior data architect, you will be responsible for designing and delivering a scalable and secure big data platform. The platform will support the enterprise's risk management initiatives by processing billions of transactions per day.This role requires strong technical leadership skills to guide the engineering organization in the design, roll out, and...

  • Senior Big Data Engineer

    11 hours ago

    Gulbarga, Karnataka, India beBeeMentor Full time ₹ 15,00,000 - ₹ 18,00,000

    Job Title: Big Data DeveloperAre you a skilled developer seeking a challenging role in the field of big data?This position is ideal for a talented individual who thrives in a fast-paced technical environment and is passionate about mentoring and developing the next generation of engineers.You will work on high-impact big data projects, provide technical...

  • Lead Big Data Engineer

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    About the Role:This position requires a highly skilled data engineer to develop efficient and scalable data pipelines, perform integration testing, and provide estimates for development, testing & deployments.Key Responsibilities:Design and implement high-quality data pipelines that support increasing data volume and complexityCollaborate with analytics and...

  • Big Data Solutions Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time US$ 7,50,000 - US$ 15,00,000

    Role SummaryWe are seeking a highly skilled Big Data Developer to join our data engineering team. As a key member of the team, you will be responsible for designing, developing, and deploying large-scale data processing systems.Key ResponsibilitiesDesign and develop scalable data processing systemsDevelop and implement efficient algorithms for data...

  • Visionary Big Data Engineer

    1 week ago

    Gulbarga, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a seasoned Big Data Solutions Architect to spearhead the development of large-scale data processing systems.The ideal candidate will have 8-15 years of experience in designing and implementing big data engineering solutions using technologies such as Apache Spark, Hadoop, Hive, and NoSQL databases.Key skills required include:

  • Senior Big Data Architect

    1 week ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Big Data Engineer OpportunityWe are seeking a highly skilled Big Data Engineer to join our data engineering team. The ideal candidate will have expertise in Scala, Spark, Python, NiFi, and Apache Kafka.The role involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems. Strong experience in cloud-based...

  • Big Data Analytics Professional

    2 days ago

    Gulbarga, Karnataka, India beBeeDataAnalytics Full time ₹ 50,00,000 - ₹ 1,00,00,000

    We are seeking an experienced Big Data Analytics professional to design and implement scalable data solutions using Google Cloud Platform services.Key Responsibilities:Design, develop, and optimize data pipelines using GCP BigQuery, Dataflow, and Apache Airflow.Utilize the Big Data Hadoop ecosystem to manage and process vast datasets efficiently.Collaborate...

  • Big Data Cloud Specialist

    4 days ago

    Gulbarga, Karnataka, India beBeeCloud Full time ₹ 36,00,000 - ₹ 48,00,000

    Job Title: Cloud Architect for Big Data SolutionsJob Description:We are seeking a highly skilled Cloud Architect with expertise in Big Data management and data quality to join our team. As a key member of our organization, you will be responsible for designing and implementing scalable cloud-based solutions that meet the needs of our clients.The ideal...