Senior Data Engineer

3 weeks ago

Hyderabad, Telangana, India Talescope Full time

Technical knowledge :

- AWS, Python, SQL, S3, EC2, Glue, Athena, Lambda, DynamoDB, RedShift, Step Functions, Cloud Formation, CI/CD Pipelines, Github, EMR, RDS,AWS Lake Formation, GitLab, Jenkins, and AWS CodePipeline.

Role Summary :

As a Senior Data Engineer,with over 5 years of expertise in Python, PySpark, SQL to design, develop and optimize complex data pipelines, support data modeling, and contribute to the architecture that supports big data processing and analytics to cutting-edge cloud solutions that drive business growth. You will lead the design and implementation of scalable, high-performance data solutions on AWS and mentor junior team members.This role demands a deep understanding of AWS services, big data tools, and complex architectures to support large-scale data processing and advanced analytics.

Key Responsibilities :

- Design and develop robust, scalable data pipelines using AWS services, Python, PySpark, and SQL that integrate seamlessly with the broader data and product ecosystem.

- Lead the migration of legacy data warehouses and data marts to AWS cloud-based data lake and data warehouse solutions.

- Optimize data processing and storage for performance and cost.

- Implement data security and compliance best practices, in collaboration with the IT security team.

- Build flexible and scalable systems to handle the growing demands of real-time analytics and big data processing.

- Work closely with data scientists and analysts to support their data needs and assist in building complex queries and data analysis pipelines.

- Collaborate with cross-functional teams to understand their data needs and translate them into technical requirements.

- Continuously evaluate new technologies and AWS services to enhance data capabilities and performance.

- Create and maintain comprehensive documentation of data pipelines, architectures, and workflows.

- Participate in code reviews and ensure that all solutions are aligned to pre-defined architectural specifications.

- Present findings to executive leadership and recommend data-driven strategies for business growth.

- Communicate effectively with different levels of management to gather use cases/requirements and provide designs that cater to those stakeholders.

- Handle clients in multiple industries at the same time, balancing their unique needs.

- Provide mentoring and guidance to junior data engineers and team members.

Requirements :

- 5+ years of experience in a data engineering role with a strong focus on AWS, Python, PySpark, Hive, and SQL.

- Proven experience in designing and delivering large-scale data warehousing and data processing solutions.

- Lead the design and implementation of complex, scalable data pipelines using AWS services such as S3, EC2, EMR, RDS, Redshift, Glue, Lambda, Athena, and AWS Lake Formation.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.

- Deep knowledge of big data technologies and ETL tools, such as Apache Spark, PySpark, Hadoop, Kafka, and Spark Streaming.

- Implement data architecture patterns, including event-driven pipelines, Lambda architectures, and data lakes.

- Experience with cloud platforms such as AWS, Azure, and GCP.

- Incorporate modern tools like Databricks, Airflow, and Terraform for orchestration and infrastructure as code.

- Implement continuous integration and delivery pipelines using GitLab, Jenkins, and AWS CodePipeline.

- Ensure data security, governance, and compliance by leveraging tools such as IAM, KMS, and AWS CloudTrail.

- Mentor junior engineers, fostering a culture of continuous learning and improvement.

- Excellent problem-solving and analytical skills, with a strategic mindset.

- Strong communication and leadership skills, with the ability to influence stakeholders at all levels.

- Ability to work independently as well as part of a team in a fast-paced environment.

- Advanced data visualization skills and the ability to present complex data in a clear and concise manner.

- Excellent communication skills, both written and verbal, to collaborate effectively across teams and levels.

Preferred Skills :

- Experience with Databricks, Snowflake, and machine learning pipelines.

- Exposure to real-time data streaming technologies and architectures.

- Familiarity with containerization and serverless computing (Docker, Kubernetes, AWS Lambda).

(ref:hirist.tech)

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Enable Data Incorporated is seeking a highly skilled Senior Data Engineer with expertise in cloud technologies, Spark, and Databricks to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients' business ecosystem.As a Senior Data...

  • 15h Left: Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Enable Data Incorporated is seeking a highly skilled Senior Data Engineer with expertise in cloud technologies, Spark, and Databricks to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients' business ecosystem.As a Senior Data...

  • Big Data Engineer, Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India data Remote Work Freelance Full time

    What can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Analysis

    2 weeks ago

    Hyderabad, Telangana, India data Remote Work Freelance Full time

    What can you tell your friends when they ask you what you do? "As a Data Analyst at data.ai, I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing...

  • Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India State Street Corporation Full time

    About Our CompanyState Street Corporation, through its FinTech division Charles River Development, has been a leader in the financial industry. Our cloud-based front office technology, Alpha Platform, enables clients to manage their investments effectively.Job Title: Data Integration Developer/Senior Data EngineerWe are seeking an experienced professional to...

  • Senior Quality Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Enable Data Incorporated is seeking a Senior Quality Engineer to join our innovative team. As a Senior Quality Engineer, you will play a pivotal role in ensuring the quality and reliability of the software products we deliver. You will leverage your expertise to design, implement, and execute comprehensive testing strategies aimed at improving the overall...

  • Senior Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Mouri Tech (P) Ltd Full time

    Company Description : MOURI Tech is a leading Global Enterprise Solutions Provider based in Hyderabad. We offer high-end innovative and agile Service and Delivery centric solutions to our clients in diverse business verticals in both private and public sectors. Our expertise lies in solving complex business problems and innovating new products, and our...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Yo HR Consultancy Full time

    Positions : Senior Data EngineerExperience : 5 to 8 years Location : HyderabadJob Descriptions : About the role :We are seeking an experienced and innovative Senior Data Engineer to lead the design, development, and implementation of robust data pipelines and solutions for our clients. The ideal candidate will possess strong technical expertise, excellent...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India Yo HR Consultancy Full time

    Positions : Senior Data EngineerExperience : 5 to 8 years Location : HyderabadJob Descriptions : About the role :We are seeking an experienced and innovative Senior Data Engineer to lead the design, development, and implementation of robust data pipelines and solutions for our clients. The ideal candidate will possess strong technical expertise, excellent...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India SAPGM Solutions Full time

    We are looking for Senior Data Engineer, send your resume to jobs@sapgm.comCompany OverviewSAPGM Solutions is a growing leader and an expert in RPO, Staffing, Consulting, Trainings & Background Verification services. With 51-200 employees and headquartered in Ameerpet, Telangana, we belong to the IT Services and IT Consulting industry.Job OverviewWe are...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India SAPGM Solutions Full time

    Job DescriptionWe are looking for Senior Data Engineer, send your resume to [Confidential Information]Company OverviewSAPGM Solutions is a growing leader and an expert in RPO, Staffing, Consulting, Trainings & Background Verification services. With 51-200 employees and headquartered in Ameerpet, Telangana, we belong to the IT Services and IT Consulting...

  • Senior Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Anblicks Full time

    Job Title:Senior Data Engineer (Azure Databricks)Location:Hyderabad / Ahmedabad (Hybrid)Experience:5+ YearsOnly candidates available for immediate joining are preferred.Job Overview:We are looking for aSenior Data Engineerwith expertise inAzure Databricksto design, develop, and optimize large-scale data solutions. The ideal candidate will have strong...

  • Senior Data Operations Lead

    3 weeks ago

    Hyderabad, Telangana, India Data Unveil Full time

    About the jobPosition Title : Senior Operations Data LeadExperience : 6 Years to 10 YearsLocation : Hyderabad, TelanganaHire Type : Full Time, On-siteStart Date : ImmediateOverview :- The Operations Data Delivery Lead is responsible for managing and overseeing the efficient delivery of data within the organization.- This role ensures that data operations and...

  • Senior Data Operations Lead

    1 week ago

    Hyderabad, Telangana, India Data Unveil Full time

    About the jobPosition Title : Senior Operations Data LeadExperience : 6 Years to 10 YearsLocation : Hyderabad, TelanganaHire Type : Full Time, On-siteStart Date : ImmediateOverview :- The Operations Data Delivery Lead is responsible for managing and overseeing the efficient delivery of data within the organization.- This role ensures that data operations and...

  • Senior Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India MaximaTek Full time

    Send applications tohrhyd@maximatek.com(or) reach me at9849143383 .Position:Senior Data EngineerType:Part-Time, RemoteExperience:Minimum 6yearsJob Summary:We are seeking a skilled and experiencedSenior Data Engineerto mentor and guide our bench candidates in securing successful placements with our clients. The ideal candidate will have a strong development...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India Kellton Full time

    Senior Data EngineerJob Location: - Hyderabad (Work from Office)Experience Level: Mid-Senior (8+ Years)About Kellton:We are a global IT services and digital product design and development company with subsidiaries that serve startup, mid-market, and enterprise clients across diverse industries, including Finance, Healthcare, Manufacturing, Retail,...

  • Senior Data Engineer

    4 days ago

    Hyderabad, Telangana, India Kellton Full time

    Senior Data Engineer Job Location: - Hyderabad (Work from Office) Experience Level: Mid-Senior (8+ Years) About Kellton: We are a global IT services and digital product design and development company with subsidiaries that serve startup, mid-market, and enterprise clients across diverse industries, including Finance, Healthcare, Manufacturing, Retail,...

  • Senior Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India NxDT Full time

    Position : Data EngineerExperience : 7+ years (Senior)Number of positions : 4Job Description : Python Developer - Data engineer is expected to work in investment banking / fund management domain with excellent coding and debugging skills in python, deep understanding of algorithms in data ingestion, transformation and storage mechanismResponsibilities : -...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India NxDT Full time

    Position : Data EngineerExperience : 7+ years (Senior)Number of positions : 4Job Description : Python Developer - Data engineer is expected to work in investment banking / fund management domain with excellent coding and debugging skills in python, deep understanding of algorithms in data ingestion, transformation and storage mechanismResponsibilities : -...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India NxDT Full time

    Position : Data Engineer Experience : 7+ years (Senior) Number of positions : 4Job Description : Python Developer - Data engineer is expected to work in investment banking / fund management domain with excellent coding and debugging skills in python, deep understanding of algorithms in data ingestion, transformation and storage mechanismResponsibilities : -...