Data Engineer

3 months ago

Vijayawada, India Material Full time

Role :- AWS Data Engineer


Location :- Gurgaon / Bangalore


Notice :- Immediate Joiners Only


Skills:- AWS , Spark , Python , Glue , S3 , Redshift etc..


Experience:- 4+ Years


Position Overview:

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have extensive experience with AWS Glue, Apache Airflow, Kafka, SQL, Python and DataOps tools and technologies. Knowledge of SAP HANA & Snowflake is a plus. This role is critical for designing, developing, and maintaining our client’s data pipeline architecture, ensuring the efficient and reliable flow of data across the organization.

Key Responsibilities:

  • Design, Develop, and Maintain Data Pipelines:
  • Develop robust and scalable data pipelines using AWS Glue, Apache Airflow, and other relevant technologies.
  • Integrate various data sources, including SAP HANA, Kafka, and SQL databases, to ensure seamless data flow and processing.
  • Optimize data pipelines for performance and reliability.
  • Data Management and Transformation:
  • Design and implement data transformation processes to clean, enrich, and structure data for analytical purposes.
  • Utilize SQL and Python for data extraction, transformation, and loading (ETL) tasks.
  • Ensure data quality and integrity through rigorous testing and validation processes.
  • Collaboration and Communication:
  • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
  • Collaborate with cross-functional teams to implement DataOps practices and improve data life cycle management.
  • Monitoring and Optimization:
  • Monitor data pipeline performance and implement improvements to enhance efficiency and reduce latency.
  • Troubleshoot and resolve data-related issues, ensuring minimal disruption to data workflows.
  • Implement and manage monitoring and alerting systems to proactively identify and address potential issues.
  • Documentation and Best Practices:
  • Maintain comprehensive documentation of data pipelines, transformations, and processes.
  • Adhere to best practices in data engineering, including code versioning, testing, and deployment procedures.
  • Stay up-to-date with the latest industry trends and technologies in data engineering and DataOps.

Required Skills and Qualifications:

  • Technical Expertise:
  • Extensive experience with AWS Glue for data integration and transformation.
  • Proficient in Apache Airflow for workflow orchestration.
  • Strong knowledge of Kafka for real-time data streaming and processing.
  • Advanced SQL skills for querying and managing relational databases.
  • Proficiency in Python for scripting and automation tasks.
  • Experience with SAP HANA for data storage and management.
  • Familiarity with Data Ops tools and methodologies for continuous integration and delivery in data engineering.
  • Preferred Skills:
  • Knowledge of Snowflake for cloud-based data warehousing solutions.
  • Experience with other AWS data services such as Redshift, S3, and Athena.
  • Familiarity with big data technologies such as Hadoop, Spark, and Hive.
  • Soft Skills:
  • Strong analytical and problem-solving skills.
  • Excellent communication and collaboration abilities.
  • Detail-oriented with a commitment to data quality and accuracy.
  • Ability to work independently and manage multiple projects simultaneously.

  • Data Engineer

    3 days ago

    Vijayawada, India Tata Consultancy Services Full time

    Greetings from TATA Consultancy ServicesJob Openings at TCSSkill :Sr. Azure Data Engineer with data bricksExp range :5+ yearsRole : Permanent RoleJob location :bangaloreCurrent location : Anywhere In IndiaInterview mode : MS Teams Pls find the Job Description below. Job Description for Sr. Data Engineer Role –Data Engineer profile with 8+ years’...

  • Data Engineer

    3 weeks ago

    Vijayawada, India Yoda Tech Full time

    Job Description: Data EngineerPosition: Data EngineerLocation: RemoteType: Full-TimeAbout Us:Yoda Tech is a leading organization dedicated to leveraging data to drive insights and innovation. We are seeking a talented Data Engineer to join our dynamic team and contribute to our data-driven initiatives.Key Responsibilities:Design, develop, and maintain robust...

  • Data Engineer

    2 months ago

    vijayawada, India MobiusEngine.AI Full time

    Our Founder and CEO: About Us:We are a dynamic and innovative company focused on leveraging data to drive decisions and optimize performance. We're seeking a talented , SELF DRIVEN Data Engineer with strong expertise in BigQuery, Looker, Python, and Google Cloud Platform (GCP) to join our growing team.Compensation: Fixed comp - Rs. 10,000 - First 4 weeks |...

  • Data Engineer

    2 months ago

    Vijayawada, India Tata Consultancy Services Full time

    Azure Data Engineer Experience Range: 4 to 13 yearsLocation : HyderabadWalk In Drive Date: 16-Nov 2024Registration Time: 09:30 AM – 12:30 PMVenue:Kohinoor Park Plot No 1, Hitech City Road, Rd Number 1, HITEC City, Hyderabad, Telangana 500084Spoc : WandaDesigning and implementing scalable and secure data processing pipelines using Azure Data Factory, Azure...

  • Senior Data Engineer

    2 weeks ago

    Vijayawada, India Spendflo Full time

    Overview:As a Senior Data Engineer, you will be responsible for building and maintaining the entire data infrastructure at Spendflo. This includes designing the data models, creating and optimizing ETL pipelines, and ensuring the scalability, reliability, and performance of our data systems. You will work closely with cross-functional teams to enable...

  • AWS Data Engineer

    3 months ago

    Vijayawada, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • Data Engineer

    2 weeks ago

    Vijayawada, India Wipro Full time

    About the role:The Enterprise Document Delivery team is seeking a Sr ETL Software Engineer who will participate in the entire system development lifecycle of applications related to document generation processing, printing, electronic and postal delivery for the bank. The team supports high volume statement applications residing on different platforms. The...

  • Lead Data Engineer

    2 weeks ago

    Vijayawada, India Cynosure Corporate Solutions Full time

    Roles and Responsibilities:Working with clients to understand their data.Based on the understanding you will be building the data structures and pipelines.You will be working on the application from end to end collaborating with UI and other development teams.You will be working with various cloud providers such as Azure & AWS.You will be engineering data...

  • Cloud Infrastructure Engineer for Data Engineering Stack

    3 weeks ago

    Vijayawada, Andhra Pradesh, India GIOS Technology Full time

    At GIOS Technology, we are seeking a highly skilled Senior Cloud Infrastructure Engineer to join our team based at Hyderabad, India.Job Overview:We are looking for an experienced Senior Cloud Infrastructure Engineer to automate infrastructure creation on Google Cloud Platform (GCP) using Terraform. The candidate will be responsible for provisioning and...

  • Data Scientist or Engineer

    3 months ago

    Vijayawada, India AKS ProTalent Full time

    We are looking for a Data Scientist to analyze large amounts of raw information to find patterns that will help improve our company. We will rely on you to build data products to extract valuable business insights. In this role, you should be highly analytical with a knack for analysis, math and statistics. Critical thinking and problem-solving skills are...

  • AWS Data Engineer

    5 days ago

    Vijayawada, Andhra Pradesh, India Programmers Full time

    About the Role:As an AWS Data Engineer in our team, you will play a crucial part in shaping the future of data-driven decision-making for our clients. We are seeking an experienced professional with a strong background in data engineering, analytics, and marketing to join our crew.Job Summary:We are looking for a talented AWS Data Engineer to lead our...

  • Data Engineer

    2 weeks ago

    Vijayawada, India Wipro Full time

    Dear Candidates,Wipro is Hiring!!!Informatica Data CatalogExperience- 6+ YearsLocation- PAN IndiaNotice Period - Immediate - 30 daysRoles & Responsibilities:Overall experience of 6+ years with good experience of working in Informatica Data Catalogue Implementation. Proven experience with Informatica Data Catalog.Strong understanding of Data Discovery,...

  • Staffbee Solutions INC | Senior Data Engineer

    2 weeks ago

    Vijayawada, India Staffbee Solutions INC Full time

    Looking for Freelance Opportunity In Data engineering?We are seeking theData Engineer Panelist @ 8+ yearsof ExperienceThis Is aFreelace_WFH/Remote_USA Shift OpportunityMust have atleast8+Years of exp at any one One cloud Tech@ GCP, AZure or AWSshould have excellent knowledge @Hadoop, python/pyspark, Bigdata,SQL & ETLRole Overview:We are seeking a highly...

  • Spotfire Administrator

    4 weeks ago

    Vijayawada, India Celito Tech, Inc. Full time

    Job Title: SPOTFIRE ADMINISTRATOR & DEVELOPER Reports To: Technical Architect (Data & Analytics), IndiaEmployment Type: FTELocation : Remote (India) Job SummaryThis role requires a skilled Data Analytics Engineer with a strong background in data analysis, data engineering, analytics development and administration of data analytics platforms.The candidate...

  • Senior/ Lead Data Engineer- SPARK/ Snowfalke

    7 months ago

    Vijayawada, India Anblicks Full time

    Lead Data Engineer :Exp : 10+Positions : 8Location : VijayawdaBachelor's or Master's degree in Computer Science, Information Systems, or a related field10+ years of experience in data engineering and data architectureStrong knowledge of data technologies, such as Snowflake, Databricks, Apache Spark, Hadoop, Dbt, Fivetran, Azure Data FactoryExperience...

  • ATE Data Analysis Engineer

    4 months ago

    Vijayawada, India Wipro Full time

    Job Title: ATE Data Analysis EngineerExperience: 5yrs-10 yrsLocation: Bengaluru, Karnataka, IndiaDescription:Experience with Data analysis tools (JMP, Datapower, O+, PDF Solutions, etc.) for Vmin, Yield, Test Time & Bin Pareto AnalysisAbility to write scripts to automate and speed up data analysis in a repeated manner with different input data sets (mainly...

  • AWS Cloud Data Engineer for Snowflake and AWS Services

    1 day ago

    Vijayawada, Andhra Pradesh, India Truelancer Full time

    Job Title:AWS Cloud Data Engineer (Snowflake and AWS)About TruelancerWe are a leading global freelance marketplace that connects businesses with skilled professionals.Salary Range:$120,000 - $200,000 per yearCompany Overview:Truelancer is a technology company that empowers businesses to find the right talent for their projects. We believe in creating a...

  • Cloud Data Engineer

    4 weeks ago

    Vijayawada, Andhra Pradesh, India HCLTech Full time

    About the Role:We are seeking a highly skilled Cloud Data Engineer to join our team at HCLTech. As a key member of our data engineering team, you will play a critical role in designing and implementing scalable solutions for our clients.Job Description:Develop and maintain end-to-end data pipelines using cloud-native solutions to extract, load, and transform...

  • Data Architect

    2 weeks ago

    Vijayawada, India Ubique Systems Full time

    We are looking for a Data Architect with:Location: Delhi NCR, Hyderabad, Bangalore, Pune, Chennai12+ Years of experience for the below:• Experience in cloud platform expertise (Azure, AWS, Google Cloud).• Strong knowledge of managing security infrastructure and applications.• Direct, hands-on experience or strong working knowledge of vulnerability...

  • Data engineer

    1 month ago

    Vijayawada, India HCLTech Full time

    Responsibilities -Expertly perform tasks under supervision.Develop and maintain end-to-end data pipelines using cloud-native solutions to extract, load, and transform data from disparate data sources to a cloud data warehouse.Capable of formatting and distributing custom data extracts through various means (e.g., custom SFTPs, APIs (e.g., RESTful), and other...