Senior Data Engineer

1 month ago

India USV PRIVATE LIMITED Full time

Overview:

As a Senior Data Engineer, you will play a pivotal role in designing, implementing, and maintaining the data infrastructure of our organization. You will be responsible for developing robust data pipelines, optimizing data workflows, and ensuring the reliability and scalability of our data systems. Collaboration with cross-functional teams including data scientists, software engineers, and business analysts will be essential to drive data-driven decision-making and support various business initiatives.

Responsibilities:

1. Data Pipeline Development:

- Design, build, and maintain scalable data pipelines to ingest, process, and transform large volumes of structured and unstructured data from diverse sources.

- Implement efficient ETL (Extract, Transform, Load) processes to ensure timely and accurate data delivery to downstream systems.

- Optimize data workflows for performance, reliability, and cost-effectiveness, leveraging technologies such as Apache Spark, Kafka, or similar distributed computing frameworks.

2. Data Modeling and Architecture:

- Develop and maintain data models, schemas, and metadata to support analytical and reporting requirements.

- Design and optimize data storage solutions including relational databases, NoSQL databases, data lakes, and data warehouses.

- Collaborate with data architects and infrastructure teams to ensure alignment with overall data architecture and best practices.

3. Data Quality and Governance:

- Implement data quality checks and validation processes to ensure data accuracy, consistency, and completeness.

- Establish and enforce data governance policies, standards, and procedures to maintain data integrity and compliance with regulatory requirements.

- Monitor data pipelines and proactively identify and resolve data quality issues or anomalies.

4. Performance Tuning and Optimization:

- Identify performance bottlenecks in data processing and storage systems and implement optimizations to improve throughput, latency, and resource utilization.

- Conduct capacity planning and scalability assessments to support growing data volumes and user demands.

- Collaborate with infrastructure teams to fine-tune hardware configurations and cloud resources for optimal performance.

5. Collaboration and Communication:

- Work closely with cross-functional teams including data scientists, software engineers, business analysts, and product managers to understand data requirements and deliver data solutions that meet business needs.

- Communicate effectively with stakeholders to gather requirements, provide project updates, and present insights derived from data analysis.

- Mentor junior data engineers and provide technical guidance and support as needed.

Qualifications:

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 8+ years of experience in data engineering roles, with a proven track record of designing and implementing data solutions at scale.

- Proficiency in programming languages such as Python, Java, or Scala, with experience in building data pipelines using frameworks like Apache Spark, Apache Beam, or similar.

- Strong SQL skills and experience working with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).

- Hands-on experience with cloud platforms such as AWS, GCP, or Azure, including services like S3, EC2, EMR, Dataflow, BigQuery, or equivalent.

- Solid understanding of data modeling concepts, data warehousing principles, and ETL best practices.

- Familiarity with data governance frameworks, data quality tools, and regulatory requirements (e.g., GDPR, HIPAA).

- Excellent problem-solving skills and the ability to work effectively in a fast-paced, dynamic environment.

- Strong communication and collaboration skills, with the ability to interact with stakeholders at all levels of the organization.

Additional Preferred Skills:

- Experience with containerization and orchestration tools such as Docker, Kubernetes.

- Knowledge of stream processing frameworks like Apache Kafka, Apache Flink, or similar.

- Familiarity with machine learning and data science concepts.

- Certification in relevant technologies or cloud platforms (e.g., AWS Certified Big Data - Specialty, Google Professional Data Engineer).

  • Data engineer

    2 weeks ago

    India Pacific Data Integrators Full time

    Job Overview We are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on...

  • Data Engineer

    3 weeks ago

    India Pacific Data Integrators Full time

    Job OverviewWe are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on experience in...

  • Data Engineer

    3 weeks ago

    India Pacific Data Integrators Full time

    Job Overview We are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on experience...

  • Data Engineer

    3 weeks ago

    India Pacific Data Integrators Full time

    Job Overview We are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on...

  • Senior data engineer

    2 weeks ago

    India Trek Health Full time

    Company Description Trek Health is revolutionizing healthcare with its Payer Price Transparency Analytics Platform. Our solutions offer payer-provider price transparency insights and analytics. We take pride in ingesting and processsing multi-terabytes of data every month and provide actionable insights to our customers. The company is backed by $3 M seed...

  • Pyspark data engineer

    2 weeks ago

    India ITI Data Full time

    Location : India Type : Full-time Experience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, Healthcare Job Description We are...

  • ITI Data | PySpark Data Engineer

    3 weeks ago

    india ITI Data Full time

    Location : India Type : Full-time Experience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, Healthcare Job Description We are looking for a PySpark...

  • ITI Data | PySpark Data Engineer

    3 weeks ago

    india ITI Data Full time

    Location : IndiaType : Full-timeExperience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, HealthcareJob DescriptionWe are looking for a PySpark solutions...

  • Senior Data Infrastructure Architect

    4 weeks ago

    India OSD Data Services Full time

    About the RoleWe are seeking a highly skilled Senior Data Infrastructure Architect to join our team at OSD Data Services. This is an exciting opportunity to work with cutting-edge technologies and be part of a fast-growing company that's transforming how businesses leverage their data.

  • Senior Data Engineer

    3 weeks ago

    India Trek Health Full time

    Company Description Trek Health is revolutionizing healthcare with its Payer Price Transparency Analytics Platform. Our solutions offer payer-provider price transparency insights and analytics. We take pride in ingesting and processsing multi-terabytes of data every month and provide actionable insights to our customers. The company is backed by $3M seed...

  • Senior Data Engineer

    3 weeks ago

    India Trek Health Full time

    Company DescriptionTrek Health is revolutionizing healthcare with its Payer Price Transparency Analytics Platform. Our solutions offer payer-provider price transparency insights and analytics. We take pride in ingesting and processsing multi-terabytes of data every month and provide actionable insights to our customers. The company is backed by $3M seed...

  • Senior Data Engineer

    3 weeks ago

    India Trek Health Full time

    Company Description Trek Health is revolutionizing healthcare with its Payer Price Transparency Analytics Platform. Our solutions offer payer-provider price transparency insights and analytics. We take pride in ingesting and processsing multi-terabytes of data every month and provide actionable insights to our customers. The company is backed by $3M seed...

  • Senior Cloud Data Engineer

    5 days ago

    India Recro Full time

    Job Title: Senior Cloud Data EngineerWe are seeking a highly skilled AWS Data Engineer to join our team as a Senior Cloud Data Engineer. The successful candidate will have 5+ years of experience in data engineering, with a strong background in AWS services and advanced proficiency in Python, Py Spark, and data pipeline architecture.This is an exciting...

  • PySpark Data Engineer

    3 weeks ago

    India ITI Data Full time

    Location : IndiaType : Full-timeExperience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, HealthcareJob DescriptionWe are looking for a PySpark solutions...

  • PySpark Data Engineer

    3 weeks ago

    India ITI Data Full time

    Location : India Type : Full-time Experience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, Healthcare Job Description We are looking for a PySpark...

  • Data Platform Engineer

    4 weeks ago

    India OSD Data Services Full time

    Data Platform EngineerLocation: RemoteType: InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...

  • Data Platform Engineer

    4 weeks ago

    India OSD Data Services Full time

    Data Platform Engineer Location : Remote Type : Internship About Us At OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve...

  • Pacific Data Integrators | Data Engineer

    2 days ago

    india Pacific Data Integrators Full time

    Job Overview We are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on experience...

  • Pacific Data Integrators | Data Engineer

    3 weeks ago

    india Pacific Data Integrators Full time

    Job OverviewWe are seeking a skilled Data Engineer to join our team. In this role, you will be responsible for managing and optimizing data pipelines, ensuring the seamless movement of data across systems, and supporting data initiatives through strong technical expertise. The ideal candidate will be proficient in SQL and Python, with hands-on experience in...

  • Senior Data Engineer

    4 weeks ago

    India Tech Mahindra Full time

    Senior Data Engineer Location; Pan India 1. ADF Minimum 5 year's experience) 2. PySpark ( 5+ years' experience ) 3. SQL (10 + years' experience).