Lead Data Pipeline Engineer

1 week ago

Hyderabad, India People Tech Group Inc Full time

Job Title: Data EngineerLocation: HyderabadExperience: 5+Years (Work from Office)Job Summary:We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects. The ideal candidate should possess strong expertise in SQL, Python, data modeling, data warehousing, and ETL pipeline development. Experience with big data tools like Hadoop and Spark, along with AWS services such as Redshift, S3, Glue, EMR, and Lambda, is essential. This role provides an excellent opportunity to work on large-scale data solutions, enabling data-driven decision-making and operational excellence.Key Responsibilities: - Design, build, and maintain scalable data pipelines and ETL processes. - Develop and optimize data models and data warehouse architectures. - Implement and manage big data technologies and cloud-based data solutions. - Perform data migration, data transformation, and integration from multiple sources. - Collaborate with data scientists, analysts, and business teams to understand data needs and deliver solutions. - Ensure data quality, consistency, and security across all data pipelines and storage systems. - Optimize performance and manage cost-efficient AWS cloud resources.Basic Qualifications: - Master's degree in Computer Science, Engineering, Analytics, Mathematics, Statistics, IT, or equivalent. - 5+ years of experience in Data Engineering and data migration projects. - Proficient in SQL and Python for data processing and analysis. - Strong experience in data modeling, data warehousing, and building data pipelines. - Hands-on experience with big data technologies like Hadoop, Spark, etc. - Expertise in AWS services including Redshift, S3, AWS Glue, EMR, Kinesis, Firehose, Lambda, and IAM. - Understanding of ETL development best practices and principles.Preferred Qualifications: - Knowledge of data security and data privacy best practices. - Experience with DevOps and CI/CD practices related to data workflows. - Familiarity with data lake architectures and real-time data streaming. - Strong problem-solving abilities and attention to detail. - Excellent verbal and written communication skills. - Ability to work independently and in a team-oriented environment.Good to Have: - Experience with orchestration tools like Airflow or Step Functions. - Exposure to BI/Visualization tools like QuickSight, Tableau, or Power BI. - Understanding of data governance and compliance standards.

  • Surescripts Integration Engineer

    4 weeks ago

    Hyderabad, India Pipeline Medical Full time

    Pipeline Medical is seeking a highly specialized Surescripts Integration Engineer to design, build, and implement a secure, fully compliant integration with the Surescripts API. This contractor will lead the end-to-end development of our internal connector that enables our platform to receive electronic prescriptions (E-Scripts) directly from the Surescripts...

  • Surescripts Integration Engineer

    4 weeks ago

    Hyderabad, India Pipeline Medical Full time

    Job Description Pipeline Medical is seeking a highly specialized Surescripts Integration Engineer to design, build, and implement a secure, fully compliant integration with the Surescripts API. This contractor will lead the end-to-end development of our internal connector that enables our platform to receive electronic prescriptions (E-Scripts) directly from...

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Surescripts Integration Engineer

    4 days ago

    Hyderabad, Telangana, India Pipeline Medical Full time

    Pipeline Medical is seeking a highly specialized Surescripts Integration Engineer to design, build, and implement a secure, fully compliant integration with the Surescripts API. This contractor will lead the end-to-end development of our internal connector that enables our platform to receive electronic prescriptions (E-Scripts) directly from the Surescripts...

  • Data Engineer

    5 days ago

    hyderabad, India beBeeDataEngineer Full time

    Role OverviewThis role offers an exciting opportunity to utilize your expertise in data processing and pipeline design to drive business growth and success. You will be responsible for developing, configuring, and optimizing Salesforce Data Processing Engine (DPE) workflows and pipelines to meet the evolving needs of our organization.The ideal candidate will...

  • Data Pipeline Architect

    2 weeks ago

    Hyderabad, India Logic Pursuits Full time

    Job Title: Data Engineer (Snowflake + dbt/Matillion)Location: Hyderabad, IndiaJob Type: Full-timeJob Description:We are looking for an experienced and results-driven Data Engineer to join our growing Data Engineering team. The ideal candidate will be proficient in building scalable, high-performance data transformation pipelines using Snowflake and dbt, and...

  • Principal Data Pipeline Engineer

    6 days ago

    Hyderabad, India Tech Mahindra Full time

    We are seeking a highly skilled and motivated Senior Data Engineer/s to architect and implement scalable ETL and data storage solutions using Microsoft Fabric and the broader Azure technology stack. This role will be pivotal in building a metadata driven data lake that ingests data from over 100 structured and semi structured sources, enabling rich insights...

  • Data Pipeline Specialist

    2 weeks ago

    hyderabad, India beBeeCloud Full time

    Job Description: We are seeking a highly skilled and experienced Unix & Cloud Data Support Engineer to join our team. This is a challenging role that requires strong technical skills, excellent communication skills, and the ability to work in a fast-paced environment.As a Unix & Cloud Data Support Engineer, you will play a crucial role in ensuring the...

  • Principal Data Pipeline Engineer

    2 weeks ago

    Hyderabad, India EverestDX Inc Full time

    About the Company:Everest DX – We are a Digital Platform Services company, headquartered in Stamford. Our Platform/Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterprise IT. Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...

  • Surescripts Integration Engineer

    2 weeks ago

    Hyderabad, Telangana, , India Pipeline Medical Full time ₹ 5,40,000 - ₹ 10,80,000 per year

    Pipeline Medical is seeking a highly specialized Surescripts Integration Engineer to design, build, and implement a secure, fully compliant integration with the Surescripts API. This contractor will lead the end -to -end development of our internal connector that enables our platform to receive electronic prescriptions (E -Scripts) directly from the...