Data Engineer

3 weeks ago

Hyderabad, Telangana, India Fraank System Full time

Job Description :


The primary responsibility involves designing, developing, and maintaining robust data pipelines leveraging AWS Glue/Pyspark and various other AWS services like S3, Redshift, Airflow, and CDK. Proficiency in Python programming is essential for this role.

Primary Skills :
  • AWS Glue/Pyspark (Minimum 3 Years): The candidate must have a strong command of AWS Glue and Pyspark, with a minimum of three years of hands-on experience. This includes designing and implementing data transformation processes, optimizing performance, and ensuring data accuracy and reliability.
  • AWS Services (S3, Redshift, Airflow, CDK - Minimum 2 Years) : Proficiency in utilizing various AWS services such as S3 for data storage, Redshift for data warehousing, Airflow for workflow management, and CDK for infrastructure as code is necessary. The candidate should have a minimum of two years of practical experience working with these services.
  • Python (Minimum 3 Years) : Strong programming skills in Python are a prerequisite for this role. The candidate should be adept at writing efficient and maintainable code for data processing, manipulation, and analysis.
Secondary Skills :
  • ETL Tools (Informatica, DataStage, etc.) : Familiarity with ETL tools like Informatica or DataStage is advantageous. The candidate should understand ETL principles and have experience in designing and implementing ETL processes to extract, transform, and load data from various sources.
  • SQL : Proficiency in SQL is essential for querying and manipulating data stored in relational databases or data warehouses. The candidate should be comfortable writing complex SQL queries and optimizing database performance.
  • Data Warehousing : A solid understanding of data warehousing concepts is required, including data modeling, schema design, and performance optimization techniques. The candidate should have experience in building and maintaining data warehouses to support business analytics and reporting needs.
Responsibilities :
  • As a Data Engineer, the candidate will be responsible for the end-to-end development and maintenance of data pipelines. This includes gathering requirements from stakeholders, designing data models, implementing data transformation processes using AWS Glue/Pyspark, and ensuring data quality and integrity. The candidate will also work closely with data scientists and analysts to understand their data requirements and provide them with curated datasets for analysis.
  • Additionally, the candidate will be responsible for optimizing and automating data processes, monitoring pipeline performance, and troubleshooting issues as they arise. The candidate should have excellent problem-solving skills and be able to work independently with minimal supervision.

(ref:hirist.tech)

  • Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Inspire Brands Hyderabad Support Center Full time

    Job Description:We are seeking a skilled and motivated Data Engineering Support Analyst to join our team. As a Data Engineering Support Analyst, you will play a crucial role in ensuring the smooth operation and support of our data engineering. Your primary responsibility will be to provide monitoring, and technical assistance, troubleshoot issues, and...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Black Turtle Full time

    A highly reputed MNC is looking for DATA ENGINEERJob Description : - As a Data Architect, you will be the key technical expert overseeing Co's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business.- As a member of the data engineering team, you will help lead the...

  • Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time

    Cognizant is hiring Data EngineersLocation: HyderabadExp: 6 years to 15 years.Employment Type: Full Time.JD:Job Description:As a Data Engineer, you will be an integral part of our data team, focusing on creating and maintaining robust data pipelines using DBT and Python. Your role will be crucial in ensuring the availability and reliability of data for...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Inspire Brands Hyderabad Support Center Full time

    Key Responsibilities:We are looking for a Data Engineer for the Enterprise Data Organization to build and manage data pipeline (Data ingest, data transformation, data distribution, quality rules, data storage etc.) for Azure cloud-based data platform. The candidate will be required to possess strong technical, analytical, programming and critical thinking...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Inspire Brands Hyderabad Support Center Full time

    Key Responsibilities:We are looking for a Data Engineer for the Enterprise Data Organization to build and manage data pipeline (Data ingest, data transformation, data distribution, quality rules, data storage etc.) for Azure cloud-based data platform. The candidate will be required to possess strong technical, analytical, programming and critical thinking...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Inspire Brands Hyderabad Support Center Full time

    We are looking for a Data Engineer for the Enterprise Data Organization to build and manage data pipeline (Data ingest, data transformation, data distribution, quality rules, data storage etc.) for Azure cloud-based data platform. The candidate will require to possess strong technical, analytical, programming and critical thinking...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Inspire Brands Hyderabad Support Center Full time

    We are looking for a Data Engineer for the Enterprise Data Organization to build and manage data pipeline (Data ingest, data transformation, data distribution, quality rules, data storage etc.) for Azure cloud-based data platform. The candidate will require to possess strong technical, analytical, programming and critical thinking...

  • Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As a Data Engineer, you will be responsible for designing and implementing complex data pipelines and analytics solutions to support key decision-making business processes in our Property & Casualty business domain. You will gain exposure to a project that is leveraging cutting edge technology that applies Big Data and Machine Learning to...

  • Engineering Manager

    2 weeks ago

    Hyderabad, Telangana, India Questsphere Private ltd. Full time

    Job Description :Skills Required : - Experience in project management and Agile/Scrum product development life cycles- Work with product management, design, support and other dev teams to create high-quality solutions- Manage best-of-class engineering teamRequirement (To be successful in this role you have) :- 10+ years of experience with technologies...

  • Senior Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the TeamJoin us as a core member of the Data Management team to develop and maintain modern data services for Swiss Re's international group of Risk Consulting specialists, who provide risk knowledge to Swiss Re clients around the world. About the RoleAs a Senior Data Engineer, you will contribute to and support data management activities using most...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Johnson & Johnson Full time

    Role : Data Engineer Location : Hyderabad Key Responsibilities: Building, maintaining & enhancing the financial systems (TM1) for budgeting/forecasting and reporting Work closely with FP&A and other finance team in determining the financial requirements, translate it to design and develop specific finance applications. Creating, enhancing, and maintaining...

  • Data Science Engineer

    7 days ago

    Hyderabad, Telangana, India JMJ CONSULTANTS Full time

    Our client is and R&D Services company head-quarterd in USA having presence in seven countries in the world. Our client is looking for a Data Scientist for Hyderabad location.Overview :As a Data Science Engineer, you will be an integral part of our cutting-edge technology team, responsible for designing, implementing, and maintaining data pipelines and...

  • Data Analytics Engineer

    3 weeks ago

    Hyderabad, Telangana, India Actalent Full time

    Description: Excellent understanding of data mining, machine learning techniques and algorithmsKnowledge of statistical programming languages, including but not limited to R, Python, SLQ, etc., to process data and gain insights from it.Strong coding experience: C/C++, Python, Java, JavaScript, etc.Solid background in database management tools such as...

  • Director of Engineering

    3 weeks ago

    Hyderabad, Telangana, India Accurate Background Full time

    Job Description :The Director of Engineering, Data will primarily drive the data strategy within the Technology organization. This person will provide domain expertise, oversee process improvements, and deliver on team projects as it relates to data initiatives including Business Intelligence, Data Architecture, Data modeling, and Data tools. The Director of...

  • Senior Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As a Senior Data Engineer, you will be responsible for designing and implementing complex data pipelines and analytics solutions to support key decision-making business processes in our Property & Casualty business domain. You will gain exposure to a project that is leveraging cutting edge technology that applies Big Data and Machine Learning...

  • Expert Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As an Expert Data Engineer, you will be responsible for leading the design and implementation of complex data pipelines and analytics solutions to support key decision-making business processes in our Reinsurance Client Solutions business domain. You will lead the development of strategic use cases within our Data Integration and Analytics...

  • Lead/Software Development Engineer

    2 weeks ago

    Hyderabad, Telangana, India MarketScope Full time

    Key Skills : - Spark Core/Streaming with Any programming language Scala/ Java/ Python- Good Understanding of Data Structures, Algorithms, Logical and problem solving skills- Data Transformation, Data Ingestion, Optimization mechanism/techniques- Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies, AWS, Postgresql.Education :...

  • Senior Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Aeries Technology Group Full time

    About Aeries :Aeries Technology is a Nasdaq listed global professional services and consulting partner, headquartered in Mumbai, India, with centers in the USA, Mexico, Singapore, and Dubai. We provide mid-size technology companies with the right mix of deep vertical specialty, functional expertise, and the right systems & solutions to scale, optimize and...

  • Data Center Regional Mechanical Engineer

    3 weeks ago

    Hyderabad, Telangana, India Amazon Full time

    As an Amazon Field Engineer, you will provide full life-cycle support to AWS Data Centers from design inception through site improvement and maintenance. You will be the 'go to' engineering resource for your region when technical advice is needed, and will use your subject matter expertise and engage with diverse teams to:• Perform design and equipment...

  • Systems Integration Specialist

    6 days ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    App Dev – Python Programmer + Good in SQL with 5+ years of Experience Cloud skills – AWS Glue, S3 and AWS DMS (nice to have) with data engineering or ETL background. Individual contributor with minimal or no hand holding required Good in communication