Data Engineer

3 weeks ago


Bangalore Urban, India Accolite Full time

Job Description: Airflow, data pipeline, ETL , Pyspark, batch processing, AWS (Mandatory)


(Immediate Joiners Only)

Exp: 4 years to 7 years

*Job Summary: *

-The Data Engineer will design, develop, and maintain scalable data pipelines and ETL processes using tools like Airflow and PySpark.

-The ideal candidate will have experience with batch processing and be proficient in AWS.

-You will collaborate with data scientists and analysts to ensure data availability and integrity for various applications.


**Key Responsibilities:** - **Data Pipeline Development:**

- Design and implement robust data pipelines to extract, transform, and load (ETL) data from various sources.

- Utilize Airflow for orchestrating complex workflows and managing data pipelines.


**Data Processing:**

- Implement batch processing techniques using PySpark to handle large volumes of data efficiently.

- Develop ELT processes to optimize data transformation within the target data warehouse.


**AWS Integration:**

- Leverage AWS services (e.g., S3, Redshift, Lambda) for data storage, processing, and orchestration.

- Ensure data security, reliability, and performance when utilizing AWS resources.


**Collaboration:**

- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide the necessary data infrastructure.

- Assist in troubleshooting and optimizing existing data workflows.


**Documentation & Best Practices:**

- Create and maintain documentation for data pipelines, workflows, and processes.

- Adhere to best practices in data engineering, including version control and monitoring.



  • Bangalore Urban, India Quantzig Full time

    Job Title: Power BI Data Engineer(BI+Azure)Experience Level: 3-5 yearsJob Type: Full-timeJob Description:Overview:We are looking for a BI & Data Engineer who will balance both business intelligence and data engineering responsibilities. In this role, you will design and implement scalable data pipelines using Azure services, while also developing powerful...


  • Bangalore Urban, India Quantzig Full time

    Job Title: Power BI Data Engineer(BI+Azure) Experience Level: 3-5 years Job Type : Full-time Job Description: Overview: We are looking for a BI & Data Engineer who will balance both business intelligence and data engineering responsibilities. In this role, you will design and implement scalable data pipelines using Azure services, while also developing...

  • Data Architect

    4 weeks ago


    Bangalore Urban, India Grid Dynamics Full time

    Experience: 12-16Years(only)Location: Bangalore/HyderabadA result- oriented thought-leader to drive the development of the data engineering practiceA trusted advisor and business partner to customers across verticals, and consulting team leader who establishes engineering processes and skill development.Responsibilities:Trusted Advisor: Provide direction to...

  • Data Architect

    4 weeks ago


    Bangalore Urban, India Grid Dynamics Full time

    Experience: 12-16Years(only) Location: Bangalore/Hyderabad A result- oriented thought-leader to drive the development of the data engineering practice A trusted advisor and business partner to customers across verticals, and consulting team leader who establishes engineering processes and skill development. Responsibilities: Trusted Advisor: Provide...

  • Data Engineer

    3 weeks ago


    Bangalore, India Amazon Full time

    Amazon Prime team is building an evergreen platform that will provide real time insights of traffic, sales, deals, prime engagement and is looking for rock start data engineer to build this. At Amazon Prime, understanding customer data is paramount to our success in providing customers with relevant and enticing benefits such as fast free shipping, instant...


  • Bangalore, India Visa Full time

    Job DescriptionPrimary ResponsibilitiesDesign: Lead the data pipeline designs as per the defined architecture of the metric repository, using/building re-usable functions and frameworks ensuring scalability, reliability and performance.Data Analysis: Understand the metric definitions, perform the analysis and lead the discussions with the requestors from...

  • Data Engineer

    6 days ago


    Bangalore, India Visa Full time

    Job DescriptionVerifi a Visa entity operates a network enabling data sharing among Acquirers, Merchants & Issuers for fraud prevention, dispute prevention and revenue recovery. Below are the key product solutions which are managed by this technology teamIntelligence Suite - Risk management platform that allows merchant to customize services to protect their...


  • bangalore, India Giant Eagle GCC Full time

    Manager Engineer Grow. Learn. Thrive. Giant Eagle is where your career can soar high. At Giant Eagle, you are so much more than an employee. You are part of the Family. About the Company Since our founding in 1931, Giant Eagle, Inc. has evolved into one of the top 40 largest private corporations in the U.S. and one of the country’s largest food...


  • Bangalore Urban, India Luxoft Full time

    Project Description:Create tooling to facilitate the development and debugging of a new calibration framework for multiple camera calibration.Responsibilities:• Good fundamental knowledge and problem-solving skills in engineering.• Hands-on, self-motivated, independent, and dedicated• Ability to work well in a diverse, collaborative and dynamic...

  • Data Engineer

    7 days ago


    Bangalore, India Visa Full time

    Job DescriptionMaster's or bachelor's degree in computer science or related fields required.Demonstrated ability in delivering high-quality, large-scale, enterprise-class data applications.Expertise in big data engineering with knowledge of Hadoop, Apache Spark, Python and SQL.Proficiency in creating and managing large-scale data pipelines and ETL...

  • Data Engineer

    1 week ago


    Bangalore, India Visa Full time

    Job DescriptionWork collaboratively with Data Analyst, Data Scientists Software Engineers and cross-functional partners to design and deploy data pipelines to deliver analytical solution.Responsible for building data pipelines, data model, data marts, data warehouse including OLAP cube in multidimensional data model with proficiency / conceptual...

  • Data Engineer

    6 days ago


    Bangalore, India Visa Full time

    Job DescriptionAt Visa, the Corporate Information Technology, Billing & Incentives Platforms team, enables Visa's revenue growth through flexible pricing engines and global revenue platforms built on next-generation technologies. This includes managing system requirements, evaluating cutting-edge technologies, design, development, integration, quality...


  • Bangalore, India Amazon Full time

    Are you interested in rapidly growing business? The Amazon ARTS team is responsible for creating core analytics tech capabilities and data engineering services for ROW (the rest of the world excl. NA and EU) countries. It comprises of platforms development, research science, software development and data engineering. ARTS develops scalable analytics...

  • Data Engineer

    4 weeks ago


    Bangalore/Hyderabad, India Talent Divas Consulting Full time

    Job Description :Are you someone looking for a challenge to work in a startup environment and innovate in a global environment to help build a SaaS Service to scale? Are you willing to roll up your sleeves and do what it takes to deliver value to Customers?As we expand our Engineering operations in India, we are looking for a Sr. Data Engineer to join us and...

  • Data Engineer 4

    3 weeks ago


    bangalore, India Porter Full time

    Porter: Creating Impactful Journeys At Porter, we're not just passionate about improving productivity; we're on a vision to Moving a billion dreams one delivery at a time. We empower businesses, both large and small, to optimize their operations and unleash unprecedented growth in their core functions. Join us in pioneering the future of last-mile...

  • Data Scientist

    4 weeks ago


    Bangalore Urban, India LTIMindtree Full time

    Job Details: Experience - 6 to 12 yrs Mandatory Skills - Gen AI, Data Science, AIML, Python, Chatbot. Location - Mumbai, Pune, Bangalore, Chennai, Hyderabad and Kolkata location. Notice - Immediate to 30 days Job Description: 7 to 10 years of experience with Data Science AIML and Generative AI technologies Strong proficiency in programming languages...


  • Bangalore, India Visa Full time

    Job DescriptionOverviewVisa Inc. is at the forefront of the GenAI revolution, aiming to establish itself as the world's leading data-driven payments company. We are seeking a hands-on Director of Data Engineering, Data Science, and GenAI/ML. This role is crucial to the development and management of our Loyalty and Marketing platforms while leveraging...

  • Data Engineer

    4 weeks ago


    Chennai/Bangalore, India SMARTWORK IT SERVICES Full time

    Job Summary : We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have hands-on experience in designing, building, and maintaining scalable data pipelines and architectures using Snowflake, Python, and Databricks. This role requires a strong foundation in data engineering practices, cloud...


  • Bangalore, Karnataka - Noida, Uttar Pradesh, India NTT DATA Full time

    About the JobNNT DATA Services is a recognized leader in IT and business services, including cloud, data, and applications, headquartered in Texas.The OpportunityYou will design, implement, and manage robust and scalable Oracle, SQL Server Applications, Database infrastructure.You will collaborate with development teams to enhance database design for optimal...


  • bangalore, India Tavant Full time

    About Tavant: With 24+ years of experience building innovative digital products and solutions, Tavant provides impactful results to its customers. It has been the frontrunner in driving digital innovation and tech-enabled transformation across a wide range of industries such as Consumer Lending, Manufacturing, Agtech, Media & Entertainment, and Retail in...