Pyspark Developer

5 days ago


Noida, Uttar Pradesh, India Birlasoft Full time ₹ 8,00,000 - ₹ 24,00,000 per year

About Birlasoft:

Birlasoft is a global technology company enabling "next-generation" digital transformation through expertise in Cloud, AI, Data, and enterprise solutions. Combining industry proficiency with advanced digital capabilities, it helps businesses accelerate change with speed, scale, and purpose, delivering "future-ready" solutions that enhance agility, resilience, and customer experience. Part of the CKA Birla Group and led by Chairman Mrs. Amita Birla, Birlasoft's nearly 12,000 professionals drive innovation while building a diverse, inclusive, and learning-oriented culture. With a strong focus on sustainability and long-term value creation, Birlasoft transforms enterprises and communities, earning its reputation as a trusted partner and one of the best places to work

About the Job :
We are seeking a highly skilled PySpark Developer with hands-on experience in Databricks . This role focuses on designing, building, and optimizing large-scale data pipelines and processing solutions on the Databricks Unified Analytics Platform. The ideal candidate will have expertise in big data frameworks, distributed computing, and cloud platforms, with a deep understanding of Databricks architecture.

Title:
Pyspark Developer

Job Location:
Multiple Location (Noida, Mumbai, Pune, Chennai, Bangalore, Hyderabad)

Educational Background:
Bachelor's degree in computer science, Management Information Systems, Mathematics or related field is strongly preferred.

Key Responsibilities:

  • Develop and manage data pipelines using PySpark on Databricks.
  • Implement ETL/ELT processes to process structured and unstructured data at scale.
  • Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks.
  • Experience in Perform Design, Development & Deployment using Azure Services (Data Factory, Databricks, PySpark, SQL)
  • Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
  • Leverage the Databricks Lakehouse architecture for advanced analytics and machine learning workflows.
  • Manage Delta Lake for ACID transactions and data versioning.
  • Develop notebooks and workflows for end-to-end data solutions.
  • Deploy and manage Databricks on Azure (e.g., Azure Databricks).
  • Use Databricks Jobs, Clusters, and Workflows to orchestrate data pipelines.
  • Optimize resource utilization and troubleshoot performance issues on the Databricks platform.
  • Deploy and manage Databricks on Azure (e.g., Azure Databricks).
  • Use Databricks Jobs, Clusters, and Workflows to orchestrate data pipelines.
  • Optimize resource utilization and troubleshoot performance issues on the Databricks platform.
  • Work closely with data scientists, data analysts, and IT teams to deliver robust data solutions.
  • Document Databricks workflows, configurations, and best practices for internal use.

Skills Required:

  • 4+ years of experience in data engineering or distributed systems development.
  • Strong programming skills in Python and PySpark.
  • Hands-on experience with Databricks and its ecosystem, including Delta Lake and Databricks SQL.
  • Knowledge of big data frameworks like Hadoop, Spark, and Kafka.
  • 4+ years of experience in data engineering or distributed systems development.
  • Strong programming skills in Python and PySpark.
  • Hands-on experience with Databricks and its ecosystem, including Delta Lake and Databricks SQL.
  • Knowledge of big data frameworks like Hadoop, Spark, and Kafka.
  • Expertise in deploying Databricks solutions Azure (e.g., Data Lake, Synapse).
  • Knowledge of Kubernetes for managing containerized workloads is advantageous
  • Experience with both SQL (e.g., PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cosmos DB).

  • Developer - PySpark

    1 week ago


    Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.Qualification:BE / B. TechLocation:Pune,Key ResponsibilitiesDesign, develop, and...


  • Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...


  • Noida, Uttar Pradesh, India Bureau Veritas Consumer Products Services Full time

    POSITION DESCRIPTIONPOSITION TITLE: Data EngineerLOCATION: Noida- IndiaTOTAL YEARS: 5-7 yearsPERSON SPECIFICATION:Summary:Looking for a Data Engineer with 57 years of experience in designing and implementing scalable data pipelines and solutions on AWS. Must be proficient in AWS Glue, PySpark, Spark SQL, S3, and RDS, with expertise in database design, ETL...

  • AWS Data Engineer

    3 days ago


    Noida, Uttar Pradesh, India Iris Software Full time ₹ 8,33,333 - ₹ 25,00,000 per year

    AWS data developers with 6-10 years experience certified candidates (AWS data engineer associate or AWS solution architect) are preferredSkills required - SQL, AWS Glue, PySpark, Air Flow, CDK, Red shiftGood communication skills and can deliver independently*Mandatory Competencies*Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS...

  • AWS Data

    3 days ago


    Noida, Uttar Pradesh, India Iris Software Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    AWS data developers with 8-10 years experience certified candidates (AWS data engineer associate or AWS solution architect) are preferredSkills required - SQL, AWS Glue, PySpark, Air Flow, CDK, Red shiftGood communication skills and can deliver independently*Mandatory Competencies*Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS...


  • Noida, Uttar Pradesh, India Tata Consultancy Services Full time ₹ 18,00,000 - ₹ 25,00,000 per year

    Greetings from TCS Walk-in Interview for Python Full Stack DeveloperExperience Range : 4- 6 yearsJob Location: BangaloreJob Description:Must-Have Skills· Strong hands-on experience with Databricks and Apache Airflow for data orchestration.· Expertise in PySpark, SQL, and Python for large-scale data processing.· Proven experience in building and...

  • Database Developer

    5 days ago


    Noida, Uttar Pradesh, India RapiPay Fintech Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Role & responsibilitiesDesign stable, reliable and effective databasesOptimize and maintain legacy systemsModify databases according to requests and perform testsLiaise with developers to improve applications and establish best practicesGather user requirements and identify new featuresDevelop technical and training manualsProvide data management support to...


  • Noida, Uttar Pradesh, India Eice Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Azure Data Engineer Microsoft Fabric SpecialistLocation: Noida, Sector 62 (Hybrid)Experience: 5+ YearsJoin: ImmediateAbout the Role:We are looking for an experienced Azure Data Engineer with strong expertise in Microsoft Fabric and modern data engineering practices. The ideal candidate will be skilled in developing scalable data pipelines,...

  • Sr Data Scientist

    21 hours ago


    Noida, Uttar Pradesh, India Birlasoft Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Country/Region: INRequisition ID: 30111Work Model:Position Type:Salary Range:Location: INDIA - NOIDA- BIRLASOFT OFFICETitle: Sr Data ScientistDescription:Area(s) of responsibilityData Scientist experience (8 to 10 years)5 years of relevant work experience as a data scientistExperience designing and building statistical forecasting models.Experience in...

  • Cloud Data Engineer

    1 week ago


    Noida, Uttar Pradesh, India SWITS DIGITAL Private Limited Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title:Cloud Data EngineerLocation:Noida, IndiaExperience:3–6 YearsAbout The RoleWe are seeking a talentedData Engineerto design, develop, and optimize enterprise-scale data platforms. The role requires strong expertise inDatabricks, PySpark, ETL pipelines, GCP services, Unity Catalog, and Terraform. The ideal candidate will collaborate with architects...