Senior Data Engineer

2 weeks ago

Jaipur, Rajasthan, India ThoughtsWin Full time

Role Overview
Design, build, and optimize large-scale, production-grade data pipelines and analytics platforms on Azure, leveraging Databricks, Synapse, and the broader Microsoft data ecosystem.

Deliver business-critical data assets for analytics, BI, and AI/ML initiatives.

Key Technical Responsibilities

  • Architect modern data lakes using Azure Data Lake Storage Gen2 for batch and streaming workloads
  • Build and maintain scalable ETL/ELT pipelines using Azure Data Factory and Databricks (PySpark, Scala, SQL)
  • Orchestrate data workflows across ADF, Databricks, and Synapse Pipelines; implement modular and reusable data pipeline components
  • Develop advanced notebooks and production jobs in Azure Databricks (PySpark, SparkSQL, Delta Lake)
  • Optimize Spark jobs by tuning partitioning, caching, cluster configuration, and autoscaling for performance and cost
  • Implement Delta Lake for ACID-compliant data lakes and enable time travel/audit features
  • Engineer real-time data ingestion from Event Hubs, IoT Hub, and Kafka into Databricks and Synapse
  • Transform and enrich raw data, building robust data models and marts for analytics and AI use cases
  • Integrate structured, semi-structured, and unstructured sources, including APIs, logs, and files
  • Implement data validation, schema enforcement, and quality checks using Databricks, PySpark, and tools like Great Expectations
  • Manage access controls : Azure AD, Databricks workspace permissions, RBAC, Key Vault integration
  • Enable end-to-end lineage and cataloging via Microsoft Purview (or Unity Catalog if multi-cloud)
  • Automate deployment of Databricks assets (notebooks, jobs, clusters) using Databricks CLI/REST API, ARM/Bicep, or Terraform
  • Build and manage CI/CD pipelines in Azure DevOps for data pipelines and infrastructure as code
  • Containerize and deploy custom code with Azure Kubernetes Service (AKS) or Databricks Jobs as needed
  • Instrument monitoring and alerting with Azure Monitor, Log Analytics, and Databricks native tools
  • Diagnose and resolve performance bottlenecks in distributed Spark jobs and pipeline orchestrations
  • Collaborate with data scientists, BI engineers, and business stakeholders to deliver data solutions
  • Document design decisions, create technical specifications, and enforce engineering standards across the team

Required Skills & Experience

  • Hands-on with :
  • Azure Data Lake Gen2, Azure Data Factory, Azure Synapse Analytics, Azure Databricks
  • PySpark, SparkSQL, advanced SQL, Delta Lake
  • Data modeling (star/snowflake), partitioning, and data warehouse concepts
  • Strong Python programming and experience with workflow/orchestration (ADF, Airflow, or Synapse Pipelines)
  • Infrastructure automation : ARM/Bicep, Terraform, Databricks CLI/API, Azure DevOps
  • Deep understanding of Spark internals, cluster optimization, cost management, and distributed computing
  • Data security, RBAC, encryption, and compliance (SOC2, ISO, GDPR/DPDPA)
  • Excellent troubleshooting, performance tuning, and documentation skills

)

  • Senior Azure Data Engineer

    2 weeks ago

    Jaipur, Rajasthan, India Datansh Solutions Full time

    Role Overview:We are looking for a Senior Azure Data Engineer who can design, build, and optimize scalable data engineering solutions on Microsoft Azure. This role involves end-to-end ownership of data pipelines, working closely with cross-functional teams, and contributing to architecture decisions while guiding junior team members. Responsibilities &...

  • Senior Data Analyst

    2 weeks ago

    Jaipur, Rajasthan, India Sanmax Full time

    Company Description Sanmax  is a Multidivisional Company of India's largest and oldest Business House Poddar Group. leader in consulting, App Building, Business Development, and outsourcing solutions. As a proven Enterprises focused on enhancing new trend of technology,Sanmax enables clients in all over india along with European countries to outperform the...

  • Senior Software Engineer- Browser Data Collection and Profiling

    1 week ago

    Jaipur, Rajasthan, India Netgraph Networking Full time

    Job Title: Senior Software Engineer Browser Data Collection & ProfilingJob Type: Full-TimeAbout UsWe are a versatile payments processing company catering to all industries. Our platform is built to handle transactions seamlessly, securely, and at scaleno matter the sector. Whether you're in retail, logistics, fintech, healthcare, or any other vertical, our...

  • Senior Software Engineer

    7 days ago

    Jaipur, Rajasthan, India Bytepx Technologies Full time

    Job Title: Senior Software Engineer (Python & SQL) Experience: 3-4 Years Location: Jaipur (Work from Office) Employment Type: Full-time Job Summary We are looking for a skilled and motivated Senior Software Engineer with strong experience in Python and SQL. The ideal candidate should be a quick learner, comfortable working with databases, deployments,...

  • Senior Engineer

    6 days ago

    Jaipur, Rajasthan, India Shree Hari Infra Projects pvt ltd Full time

    Company DescriptionSince 2007, Shree Hari Infra Projects Pvt Ltd has been committed to delivering high-quality water infrastructure solutions. The company specializes in the creation of Water Treatment Plants (WTPs), Clear Water Reservoirs (CWRs), Elevated Water Tanks (EWTs), and water distribution networks. Through its efforts, Shree Hari Infra Projects has...

  • Data Engineer

    1 week ago

    Jaipur, Rajasthan, India Celebal Technologies Full time

    Job Role – Data EngineerJob Location – JaipurJob Type – PermanentExperience Required years)Job Summary: We're looking for a talented Azure Data Engineer to solve complex data challenges for our clients using Microsoft Fabric. You will collaborate with project teams to deliver innovative data solutions that drive business value.Responsibilities:•...

  • Data Engineer

    1 week ago

    Jaipur, Rajasthan, India Marktine Full time

    Job Summary:We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks on any cloud platform (AWS, Azure, or GCP), along with proven skills in SQL, Python, and PySpark. The ideal candidate will have a solid background in database design, data processing, and building scalable data pipelines in a cloud-based environment.Key...

  • Senior Data Scientist

    2 weeks ago

    Jaipur, Rajasthan, India ATV Tech Solutions Full time

    (Senior) Data Scientist AI/ML/GenAI with 5–8 yrs experience. Strong in Python, Pandas, NumPy, Scikit-learn, PyTorch, TensorFlow, LLMs, RAG, MLOps, AWS/Azure/GCP, Snowflake. On-site Jaipur, Rajasthan.

  • Azure Data Engineer

    7 days ago

    Jaipur, Rajasthan, India Spaulding Ridge Full time

    Spaulding Ridge is an advisory and IT implementation firm. We help global organizations get financial clarity into the complex, daily sales, and operational decisions that impact profitable revenue generations, efficient operational performance, and reliable financial management.At Spaulding Ridge, we believe all business is personal. Core to our values is...

  • Data Engineer

    2 weeks ago

    Jaipur, Rajasthan, India UiPath Full time

    Life at UiPathThe people at UiPath believe in the transformative power of automation to change how the world works. We're committed to creating category-leading enterprise software that unleashes that power.To make that happen, we need people who are curious, self-propelled, generous, and genuine. People who love being part of a fast-moving, fast-thinking...