Data Engineer
1 week ago
We are at the forefront of developing innovative healthcare analytics solutions. Our vision is to create robust, scalable data pipelines that empower nationwide analytics and support our machine learning initiatives.Key ResponsibilitiesData Pipeline Engineering: Design, develop, and maintain high-performing ETL pipelines using Python (Pandas, PySpark) and SQL, orchestrated with Airflow (MWAA). Ensure seamless integration with our data lakehouse infrastructure.Data Lakehouse Management: Develop and maintain our SAIVA Data Lakehouse on AWS, focusing on quality, governance, scalability, and accessibility. Implement efficient data storage and retrieval strategies.Distributed Data Processing: Run and optimize distributed data processing jobs with Spark on AWS EMR and/or EKS. Leverage cloud-native services for efficient data processing and analysis.Data Ingestion Frameworks: Implement batch and streaming ingestion frameworks (APIs, databases, files, event streams). Ensure real-time data availability and reliability.Validation and Quality Assurance: Enforce rigorous validation and quality checks to guarantee reliable analytics and ML readiness. Develop and implement effective data quality metrics.Pipeline Monitoring and Troubleshooting: Monitor and troubleshoot pipelines with CloudWatch, integrating observability tools like Grafana, Prometheus, or Datadog. Ensure proactive issue detection and resolution.Infrastructure Provisioning: Automate infrastructure provisioning with Terraform, following AWS best practices. Ensure scalable and secure infrastructure deployment.Database Integrations: Manage SQL Server, PostgreSQL, and Snowflake integrations into the Lakehouse. Develop and maintain efficient database connections and queries.On-Call Support: Participate in an on-call rotation to support pipeline health and resolve incidents quickly. Collaborate with cross-functional teams for prompt issue resolution.Code Reviews and Engineering Best Practices: Write production-grade code, and contribute to design/code reviews and engineering best practices. Foster a culture of continuous improvement and knowledge sharing.About UsWe're passionate about leveraging data-driven insights to drive business success. Our team collaborates closely to deliver innovative solutions that meet the evolving needs of our organization.Why Join Us?Contribute to cutting-edge projects that impact business outcomes.Collaborate with a talented team of professionals who share your passion for data-driven insights.Develop your skills and expertise through ongoing training and mentorship opportunities.
-
Data Engineer
6 days ago
Surat, India de facto Infotech Full timeData Engineer – Azure / Microsoft FabricRole SummaryAs a Data Engineer with 4–5 years of experience, you will design, implement, and maintain scalable, production‑grade data pipelines and data platform solutions. Your work will deliver reliable, high-quality data for analytics, reporting, ML/AI initiatives — enabling data-driven decision making...
-
Data Science Trainer
1 week ago
Surat, India Blue Data Consulting & IT services Pvt Ltd Full time**Job Description for Data Science Trainer & Consultant** - To teach Python, R Programming, Machine learning, Deep Learning, NLP, Computer Vision, Reinforcement Learning. - Deliver corporate and retail training programs on Machine Learning, Deep Learning, Computer Vision, NLP, reinforcement learning, Algorithms, Time series and forecasting, SQL. - Work as a...
-
Data Engineering Expert
7 days ago
surat, India beBeeDataEngineering Full timeJob Title: Data Engineering ExpertWe are seeking an experienced data engineering expert with a strong background in modern data engineering tools and platforms. The ideal candidate will have a solid foundation in data integration, transformation, and migration, as well as expertise in working on complex projects.Key Requirements:3+ years of experience in...
-
Data Engineering Leader
3 days ago
surat, India beBeeData Full timeJob OverviewAs a seasoned Lead Data Engineer, you will spearhead the design and development of a cutting-edge data warehouse on AWS.This role is instrumental in building a scalable, secure, and efficient data platform that supports analytics, reporting, and AI use cases across the organization.The ideal candidate should be both technically proficient and...
-
Data Engineering Opportunity
7 days ago
surat, India beBeeDataEngineer Full timeData Engineering RoleAs a Data Engineer, you will play a key role in designing and developing robust data pipelines to support business growth.You will be working with large datasets and complex queries to ensure data accuracy and integrity.Developing ETL/ELT pipelines using Lakehouse architecture to manage structured and unstructured data.Ensuring data...
-
Data Engineering Leader
1 week ago
surat, India beBeeWebCrawler Full timeData Engineering Manager – Web CrawlingLeads data engineering and web crawling teams to deliver high-quality results.Main Responsibilities:Architect and implement scalable data pipelines using Airflow or Celery for efficient data ingestion, transformation, and storage.Design and optimize robust crawling systems using modern frameworks, handling IP...
-
Chief Data Engineer
1 week ago
surat, India beBeeDataEngineer Full timeCustomer Engineer RoleThe primary responsibility of this position is to serve as the technical point of contact for our customers. As a trusted partner, you will work closely with various teams internally and externally to deliver on complex customer use cases.This role requires a strong understanding of data engineering principles and experience in managing...
-
Data Engineering Specialist
1 week ago
surat, India beBeeDataEngineering Full timeData Engineering SpecialistWe are seeking a highly skilled Data Engineering Specialist to join our team. The ideal candidate will design, build, optimize, and manage large-scale data pipelines that support both batch and real-time analytics.Responsibilities:Implement and support real-time data streaming using Apache Kafka, Apache Spark Streaming, or...
-
Data Engineer
2 weeks ago
surat, India Infilon Technologies Pvt ltd Full timeInfilon Technologies is hiring an experienced Data Engineer (Immediate Joiner) for our US-based client, CalcuQuote. You'll play a key role in building and optimizing data pipelines, ensuring data is clean, secure, and accessible for analytics and operations. This role offers the chance to work closely with talented teams, solve real-world data challenges,...
-
Data Engineering Expert
1 week ago
surat, India beBeeDataEngineer Full timeJob OpportunityWe are seeking a highly skilled Data Engineer to join our team. This is a unique opportunity to work with cutting-edge data engineering solutions using Palantir Foundry tools.This role requires strong programming skills in Python, SQL, and optionally Java/Scala.The ideal candidate should have hands-on experience with Palantir Foundry tools...
