Senior Data Engineer
1 week ago
Description :
About the Project :
We are developing a data-driven platform designed to help companies optimize promotional activities for maximum business impact.
The solution collects and validates data, analyzes promotion effectiveness, plans promotional calendars, and integrates seamlessly with existing enterprise systems.
By leveraging machine learning, the platform enhances vendor collaboration, enables better negotiation, and empowers organizations to make informed, ROI-driven decisions.
Project Phase : Ongoing Team : Large, cross-functional team with diverse technical and business roles.
Key Responsibilities :
- Design, build, and optimize scalable and reusable data pipelines using Databricks, Spark, and SQL
- Develop and orchestrate automated ETL workflows using Airflow, Azure Data Factory (ADF), and CI/CD pipelines
- Implement and manage Delta Lake components including Delta Live Tables, Unity Catalog, and Delta Sharing
- Ensure data quality, consistency, and compliance with governance and security standards
- Collaborate with stakeholders to translate business requirements into robust technical solutions
- Optimize data systems for performance, reliability, and cost-efficiency in cloud environments
- Apply software engineering best practices including testing, modularization, refactoring, and automation
- Evaluate and implement new tools to improve pipeline automation, monitoring, and efficiency
- Lead data engineering initiatives and mentor junior team members
Required Skills & Experience :
- Bachelors or Masters degree in Computer Science, Engineering, or related field
- 7 years of hands-on experience in Data Engineering with strong Big Data expertise
- Deep expertise with Databricks, Spark, and SQL (including Spark SQL)
- Strong background in data modeling and ETL development for large-scale datasets
- Proven experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL Warehouse (server & serverless)
- Proficiency in Python for data engineering workflows
- Experience orchestrating pipelines using Airflow and implementing CI/CD with tools like Azure DevOps or GitHub Actions
- Familiarity with analytics engineering and DBT
- Knowledge of data governance, compliance, and security in cloud-based environments
- Exposure to AI/ML frameworks for workflow optimization
- Excellent communication, problem-solving, and leadership abilities
Technology Stack :
- Core : Databricks, SQL, Spark, Python, Delta Lake (Delta Live Tables, Unity Catalog, Delta Sharing)
- Orchestration : Airflow, Azure Data Factory (ADF)
- CI/CD : Azure DevOps / GitHub Actions (or equivalent)
- Cloud Platform : Azure (preferred)
- Analytics & BI : DBT and related frameworks
-
Senior Data Engineer
2 weeks ago
Multiple Locations, India DoctusTech Full timeDescription : Role Overview : We are seeking an experienced Senior Data Engineer with a strong background in data security, privacy, and compliance to join our growing engineering team. This individual will play a critical role in designing, building, and maintaining secure data pipelines, ensuring end-to-end compliance across all data systems. The...
-
Senior Data Engineer
2 days ago
Multiple Locations, India Digihelic Solutions Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per yearDescription : We are seeking a highly experienced and technically adept Senior Data Engineer to join our TAVS team in Pune. The ideal candidate will have a minimum of 12 years of relevant experience and a strong background in designing, developing, and optimizing large-scale data pipelines and data warehouse solutions. This role requires deep expertise...
-
Lead Data Engineer
5 days ago
Multiple Locations, India CAPITALNUMBERS INFOTECH LIMITED Full time ₹ 20,00,000 - ₹ 25,00,000 per yearSenior Data Engineer We are seeking a highly skilled Senior Data Engineer to strengthen and optimize our data infrastructure, which serves as the backbone of our business operations. The ideal candidate will bring deep technical expertise in data engineering, with hands-on experience across PostgreSQL, AWS (Redshift, S3), Apache Airflow, and dbt, and a...
-
Senior Data Engineer
1 week ago
Multiple Locations, India Hunarstreet Technologies Pvt. Ltd. Full time ₹ 6,00,000 - ₹ 18,00,000 per yearShift : 2:00 PM - 11:00 PM ISTExperience : 6 years of hands-on Data Engineering experienceAbout the Role : We are looking for experienced Data Engineers who can design, build, and optimize large-scale data pipelines. This role is for individual contributors who love coding, problem-solving, and working with cutting-edge big data technologies.Key...
-
Senior BI Engineer/Data Architect
2 days ago
Multiple Locations, India HNR Tech Full time ₹ 15,00,000 - ₹ 25,00,000 per yearDescription : About the Role : Were looking for an experienced Senior BI Engineer / Data Architect to lead and deliver an end-to-end BI project for one of our clients over a three-month period. This role involves full ownership of the BI architecture and delivery from designing the semantic model and building the Operational Data Store (ODS) to...
-
Senior Data Engineer
2 weeks ago
Multiple Locations, India Apidel Technology Full time ₹ 20,00,000 - ₹ 25,00,000 per yearKey Responsibilities : - Design, build, and maintain data pipelines using Azure Data Factory, Databricks, and Data Lake. - Develop efficient, reusable, and scalable ETL/ELT processes in Python, PySpark, and SQL. - Implement and maintain data models, transformations, and integrations for large-scale data platforms. - Ensure data quality,...
-
Senior Data Warehouse Engineer
3 days ago
Multiple Locations, India Norstella Full time ₹ 12,00,000 - ₹ 36,00,000 per yearDescription : At Norstella, our mission is simple : to help our clients bring life-saving therapies to market quickerand help patients in need. Founded in 2022, but with history going back to 1939, Norstella unites best-in-class brands to help clients navigate the complexities at each step of the drug development life cycle and get the right...
-
Senior Engineer
2 days ago
Multiple Locations, India INTRAEDGE TECHNOLOGIES PRIVATE LIMITED Full time ₹ 15,00,000 - ₹ 25,00,000 per yearDescription : Seeking a hands-on Senior Engineer with proven expertise in Artificial Intelligence (AI/ML) and Java development. This role combines deep technical skills with the ability to design, build, and optimize AI-powered enterprise applications. The engineer will be responsible for developing scalable Java-based systems, integrating AI/ML...
-
Data Engineer
3 days ago
Multiple Locations, India Dash Hire Full time ₹ 8,00,000 - ₹ 25,00,000 per yearDescription : The Data Engineer will be responsible for designing, developing, and maintaining the data infrastructure for a healthcare organisation. The ideal candidate will have experience in working with healthcare data, including EHR, HIMS, PACS, and RIS. They will also have experience with SQL, Elasticsearch, and data integration tools like Talend....
-
Data Engineer
2 days ago
Multiple Locations, India iTheme Consulting Full time ₹ 6,00,000 - ₹ 18,00,000 per yearPosition Overview We are seeking a skilled and proactive Data Engineer to join our growing Data & Analytics team. This role is critical to our enterprise-wide data transformation journey, enabling scalable, secure, and high-quality data pipelines that power analytics, governance, and decision-making across the organization. The Data Engineer will work...