Senior Data Engineer

3 days ago


Vadodara, Gujarat, India Reflik Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Title: Senior Data Engineer

Basic Function

The Senior Data Engineer should be an expert familiar with all areas of data warehousing technical components (e.g., ETL, Reporting, Data Model), connected infrastructure, and their integrations. The ideal candidate will be responsible for developing the overall architecture and high-level design of the data schema environment. The candidate must have extensive experience with Star Schemas, Dimensional Models, and Data Marts. The individual is expected to build efficient, flexible, extensible, and scalable ETL design and mappings. Excellent written and verbal communication skills are required as the candidate will work very closely with diverse teams. A wide degree of creativity and latitude is expected. This position reports to the Manager of Data Services.

Typical Requirements

Requires strong technical and analytical skills, data management expertise, and business acumen to achieve results. The ideal candidate should be able to deep dive into data, perform advanced analysis, discover root causes, and design scalable long-term solutions using Databricks, Spark, and related technologies to address business questions. A strong understanding of business data needs and alignment with strategic goals will significantly enhance effectiveness. The role requires the ability to prepare high-level architectural frameworks for data services and present them to business leadership. Additionally, the candidate must work well in a collaborative environment while performing a variety of detailed tasks daily. Strong oral and written communication skills are essential, along with expertise in application design and a deep understanding of distributed computing, data lake architecture, and relational database concepts. This position requires the ability to leverage both business and technical capabilities regularly.

Essential Functions

  1. Gather, structure, and process data from various sources (e.g., transactional systems, third-party applications, cloud-based financial systems, customer feedback, etc.) using Databricks and Apache Spark to enhance business insights.
  2. Develop and enforce standards, procedures, and quality control measures for data analytics in compliance with enterprise policies and best practices.
  3. Partner with business stakeholders to build scalable data models and infrastructure, leveraging Databricks' Delta Lake, MLflow, and Unity Catalog.
  4. Identify, analyze, and interpret complex data sets to develop insightful analytics and predictive models.
  5. Utilize Databricks to design and optimize data processing pipelines for large-scale data ingestion, transformation, and storage.
  6. Ensure data infrastructure completeness and compatibility to support system performance, availability, and reliability requirements.
  7. Architect and implement robust data pipelines using PySpark, SQL, and Databricks Workflows for automation.
  8. Provide input on technical challenges and recommend best practices for data engineering solutions within Databricks.
  9. Design and optimize data models for analytical and operational use cases.
  10. Develop and implement monitoring, alerting, and logging frameworks for data pipelines.
  11. Lead the architecture and implementation of next-generation cloud-based data solutions.
  12. Build scalable and reliable data integration pipelines using Databricks, SQL, Python, and Spark.
  13. Mentor and develop junior team members, fostering a data-driven culture within the organization.
  14. Develop high-quality, scalable data solutions to support business intelligence, analytics, and data science initiatives.
  15. Interface with technology teams to extract, transform, and load (ETL) data from diverse data sources into Databricks.
  16. Continuously improve data processes, automating and simplifying workflows for self-service analytics.
  17. Work with large, complex data sets to solve non-routine analysis problems, applying advanced machine learning and data processing techniques as needed.
  18. Prototype, iterate, and scale data analysis pipelines, advocating for improvements in Databricks data structures and governance.
  19. Collaborate cross-functionally to present findings effectively through data visualizations and executive-level presentations.
  20. Research and implement advanced analytics, forecasting, and optimization methods to drive business outcomes.
  21. Stay up to date with industry trends and emerging Databricks technologies to enhance data-driven capabilities.

Specialized Skills or Technical Knowledge

  • Bachelor's degree or higher in a quantitative/technical field (e.g., Computer Science, Statistics, Engineering). A Master's degree in Computer Science, Mathematics, Statistics, or Economics is preferred.
  • 5+ years of experience in data engineering, business intelligence, or data analytics, with a focus on Databricks and Apache Spark.
  • Extensive experience with SQL and Python for developing optimized queries and data transformations.
  • Expertise in Databricks ecosystem, including Delta Lake, ML flow, and Databricks SQL.
  • Experience in designing and implementing ETL/ELT pipelines on Databricks using Spark and cloud-based data platforms (Azure Data Lake, AWS S3, or Google Cloud Storage).
  • Strong data modeling, data warehousing, and data governance knowledge.
  • Experience in working with structured and unstructured data, including real-time and batch processing solutions.
  • Familiarity with data visualization tools such as Power BI, Tableau, or Looker.
  • Deep understanding of distributed computing, scalable data architecture, and cloud computing frameworks.
  • Hands-on experience with CI/CD pipelines, Infrastructure as Code (IaC), and DevOps practices in data engineering.
  • Proven track record of working with cross-functional teams, stakeholders, and senior management to deliver high-impact data solutions.
  • Knowledge of machine learning and AI-driven analytics is a plus.
  • Strong problem-solving skills and the ability to work independently and in a team-oriented environment.
  • Excellent communication skills and ability to convey complex data concepts to non-technical stakeholders.
  • Experience in a franchised organization is a plus.


  • Vadodara, Gujarat, India Techno Compass Consulting Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We're hiring a Senior Data Engineer (6+ yrs, Abu Dhabi onsite) to design and manage scalable data pipelines, models, and cloud solutions using Python, SQL, Airflow, NoSQL, Cypher queries and AWS/GCP/Azure.

  • Sr./Data Engineer

    19 hours ago


    Vadodara, Gujarat, India Imriel Technology Solutions Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for youWe at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will...

  • Senior AI Engineer

    1 week ago


    Vadodara, Gujarat, India P5Infosoft Full time ₹ 3,87,000 - ₹ 12,13,594 per year

    Job Title: Senior AI EngineerLocation: VadodaraJob Type: Full TimeAbout Us:PATEL Processing Pvt Ltd is a leading IT organization dedicated to leveraging cutting-edge technology to drive innovation and improve outcomes. We are seeking a talented and experienced Senior AI Engineer to join our dynamic team and contribute to the development of advanced AI...

  • Lead Data Engineer

    1 week ago


    Vadodara, Gujarat, India Asian Hires Full time ₹ 24,00,000 - ₹ 36,00,000 per year

    Reporting RelationshipPosition Reports To: Senior Manager Application DevelopmentDepartment: Information TechnologyPosition Location: Vadodara, Gujarat, India (Hybrid)Budget: 20,00,000 -30,00,000 PA.Job SummaryWe are seeking a strategic and technically proficient Lead Data Warehouse Engineer to architect, manage, and evolve our enterprise data warehouse....

  • Senior Engineer

    2 weeks ago


    Vadodara, Gujarat, India Lucy Electric Full time US$ 80,000 - US$ 1,20,000 per year

    Internal Job Title:Senior Design Engineer (Medium Voltage)Company:Lucy Electric Manufacturing & Technologies IndiaLocation:Halol, Vadodara, Gujarat, IndiaJob Reference No:3012We are looking to hire aSenior Design Engineer to support new product engineeringactivities. This will include concept development, detailed design & development of new products,...

  • Senior Engineer

    2 weeks ago


    Vadodara, Gujarat, India Gentoro People Consulting Full time ₹ 84,000 - ₹ 15,00,000 per year

    Job Title: Senior Engineer – Offshore (FPSO)Location: Vadodara, IndiaExperience: 10+ YearsJob Summary:We are looking for a Senior Engineer with strong experience in offshore engineering, ideally from the FPSO sector. The candidate must be well-versed in international offshore standards such as DNV, ABS, API, AISC, and proficient in engineering software...


  • Vadodara, Gujarat, India Velji Ratna Sorathia Infra Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Company DescriptionVelji Ratna Sorathia Infra Pvt. Ltd. specializes in Construction, EPC, and Infrastructure projects, including Roads, Dams, Canals, WTP, STP, ETP, Pipeline Networks, Industrial Areas, Buildings, Landscaping, and Railways. We are committed to delivering high-quality infrastructure solutions and are renowned for our expertise and dedication...

  • Data Engineer

    1 week ago


    Vadodara, Gujarat, India Atlas Copco Group Full time ₹ 8,00,000 - ₹ 18,00,000 per year

    Your roleAbout UsAtlas Copco is a Swedish multinational industrial company that was founded in 1873 headquartered in Nacka, Sweden. We are world leaders in Compressors, We also manufactur industrial tools and equipments.We are an IT Service Provider company within Atlas Copco Group called Global IT Hub (India). We are also focusing on Digital transformation...


  • Vadodara, Gujarat, India Lucy Electric Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Internal Job Title:Senior Design EngineerBusiness:Lucy Electric Manufacturing & Technologies IndiaLocation:Halol, Vadodara, Gujarat, IndiaJob Reference No:3697We are looking to hire aSenior Design Engineer to support new product engineeringactivities. This will include concept development, detailed design & development of new products, validation by...

  • Senior Site Engineer

    22 hours ago


    Vadodara, Gujarat, India Vaibhav Corporation Pvt. Ltd. Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionVaibhav Corporation Pvt. Ltd. is a trusted real estate developer in Vadodara, Gujarat, specializing in building infrastructure-ready plots, super premium villas, and luxurious 4 BHK and 5 BHK residences. Approved by VMSS, Vaibhav Corporation is dedicated to delivering quality and excellence in every project.Role DescriptionThis is a...