Pyspark Developer
4 days ago
PySpark Developer – Bhopal (Onsite – no hybrid)
Experience: 5+ years
Key Responsibilities
- Develop and maintain PySpark pipelines in Azure Databricks for batch and streaming data.
- Implement ETL/ELT workflows integrating Azure Data Lake, Synapse, and Delta Lake.
- Write efficient Python and SQL code for transformations, joins, aggregations, and validations.
- Optimise Spark jobs with partitioning, caching, and tuning to reduce runtime and cost.
- Integrate pipelines into Azure Data Factory with CI/CD deployment via Git/DevOps.
- Apply monitoring, logging, and documentation standards for reliable production pipelines.
Key Requirements (Technical & Delivery Skills)
- 5+ years' hands-on experience in PySpark development within the Azure ecosystem.
- Mastery of:
- o Azure Databricks (PySpark, Delta Lake, Spark SQL)o Azure Data Factory for orchestrationo Azure Synapse Analytics for data integration/consumption
- Experience with Delta Lake/Parquet for scalable storage and analytics.
- Skilled in debugging and optimising Spark jobs for performance and cost efficiency.
- Familiar with CI/CD using Azure DevOps, Git versioning, and Agile delivery.
- Proven track record delivering production-grade pipelines in enterprise environments.
Job Type: Full-time
Work Location: In person
-
Pyspark developer intern
10 hours ago
Bhopal, Madhya Pradesh, India SAA Consultancy Full time ₹ 4,00,000 - ₹ 8,00,000 per yearKey ResponsibilitiesAssist in building PySpark ETL pipelines for structured/unstructured healthcare datasets.Support Delta Lake design ensuring schema evolution and compliance.Work with mentors on performance tuning and job orchestration.Document lineage, data quality checks, and compliance processes.Contribute to CI/CD setup and Git-based...
-
Pyspark Developer Intern
12 hours ago
Bhopal, Madhya Pradesh, India SAA Consultancy Full time ₹ 4,00,000 - ₹ 12,00,000 per yearKey ResponsibilitiesAssist in building PySpark ETL pipelines for structured/unstructured healthcare datasets.Support Delta Lake design ensuring schema evolution and compliance.Work with mentors on performance tuning and job orchestration.Document lineage, data quality checks, and compliance processes.Contribute to CI/CD setup and Git-based...
-
AI Data Engineer
4 days ago
Bhopal, Madhya Pradesh, India Lumenore Full time ₹ 8,00,000 - ₹ 24,00,000 per yearAI Data EngineerAbout the RoleWe are looking for an AI Data Engineerwho can build scalable, high-quality data pipelines that power advanced analytics and AI/ML models. If you enjoy turning raw data into reliable, production-ready datasets for AI use cases, you'll thrive in this role.What You'll DoDesign, build, and maintain end-to-end data pipelines for...
-
Azure data Brick Engineer
2 weeks ago
Bhopal, India Tata Consultancy Services Full timeRole: Azure data brick EngineerExperience: 8+Location: Pan IndiaMust-Have 8+ years of hands-on experience in development. Azure – cloud technology – Function Apps,AWS, Python, PySpark , Databricks, Azure DevOps, Synapse, API. Strong knowledge Incident, problem, Change management, service management, service request. Must have hands on experience code...
-
AIML Engineer
2 weeks ago
Bhopal, India Tata Consultancy Services Full timeCompetencies (Technical/Behavioral Competency) Must-Have** (Ideally should not be more than 3-5) AI/ML, Azure ML Studio, AI/ML On Databricks, Python & CICD Devops. Supervised and unsupervised ML and Predictive Analytics using Python • Feature generation through data exploration and SME requirements • Relational database querying • Applying...
-
Data Engineer
3 days ago
Bhopal, India Whatjobs IN C2 Full timeRole: Data Engineering Remote Looking for 5-10 years of Exp - Design, develop & maintain ETL/ELT pipelines - Ingest & transform data from APIs, DBs, files, streams - Build real-time & batch processing solutions - Data validation, quality & cleansing - Translate business needs into data models - Ensure data security, access control & compliance - Maintain...
-
Data Engineer
3 days ago
Bhopal, India Whatjobs IN C2 Full timeJob Responsibilities Design and Develop Data Pipelines: Develop and optimise scalable data pipelines using Microsoft Fabric , including Fabric Notebooks , Dataflows Gen2 , Data Pipelines , and Lakehouse architecture . Work on both batch and real-time ingestion and transformation. Integrate with Azure Data Factory or Fabric-native orchestration for smooth...
-
DE Architect
2 weeks ago
Bhopal, India Tredence Inc. Full timeRole: Databricks Technical Lead/ DE Architect Experience: 8-13 years Location: Bangalore, Chennai, Delhi, Pune, Kolkata About Tredence: Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies across industries are engaging...
-
Data Engineer
2 weeks ago
Bhopal, India Philodesign Technologies Inc Full timeJob Title:Data Engineer Experience:6–8 Years Work Mode:Remote (9 AM – 5 PM EST) Employment Type:Full-time Notice Period:Immediate JoinerAbout the Role We are seeking an experiencedData Engineerto design, develop, and implement robust data exchange and processing solutions using Azure technologies. The ideal candidate will have strong hands-on expertise...
-
Senior Data Integration Engineer
4 weeks ago
Bhopal, India Whatjobs IN C2 Full timeJob Title - Senior Data Integration Engineer Experience - 6+Years Job Summary We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intelligence ecosystem. In this critical role, you will be the subject matter expert responsible for designing and...
-
Sr. Data Engineer
2 weeks ago
Bhopal, India xponent.ai Full timeWe are an Australian Data and AI company and are expanding ourDatabricks Capability in Indiaand are seeking aSenior Databricks Data Engineerto build scalable, production-grade data platforms that enable AI and advanced analytics for our global clients. We are open, transparent, no-nonsense, innovative problem solvers and eternal learners as a team where...
-
Senior Data Integration Engineer
3 days ago
Bhopal, India Whatjobs IN C2 Full timeJob Role: Senior Data Integration Engineer Exp Required: Atleast 6 Years of relevant work experience in Data Integration Engineer with (Salesforce, Databricks & MuleSoft) Job Location: Chennai - Onsite (Initially remote for first few months, ready to work in Chennai location from Jan 2026) Company Name: Indsafri Notice Period: Immediate to 30 days SHOULD...
-
Azure Data Engineer
1 week ago
Bangalore; Bhopal; Chennai; Gurgaon; Hyderabad; Jaipur; Pune, India APAC Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAbout XebiaXebia is a trusted advisor in the modern era of digital transformation, serving hundreds of leading brands worldwide with end-to-end IT solutions. The company has experts specializing in technology consulting, software engineering, AI, digital products and platforms, data, cloud, intelligent automation, agile transformation, and industry...