Data Engineer

1 month ago


Bangalore, Karnataka, India ARR Recruitment Solutions Full time

Job Description :

We are looking for a highly skilled Senior Data Engineer with extensive experience in data integration, processing, and application development for handling large-scale data sets measured in Terabytes from multiple sources.

- The ideal candidate will have a strong background in AWS services, particularly Lambda, EMR, and Glue, along with advanced proficiency in SQL and Python.

- The candidate should be adept at building secure, scalable, and robust data solutions and should be comfortable working in self-organizing Agile teams with minimal supervision.

Responsibilities :

- Develop and maintain data processing pipelines to integrate and process large volumes of data from diverse sources.

- Implement product features that meet high standards of security, scalability, and reliability.

- Collaborate with cross-functional teams to understand requirements and design efficient data solutions.

- Take ownership of code quality and ensure that implemented features and bug fixes work seamlessly at cloud scale.

- Work within self-organizing Agile teams to deliver features and enhancements on schedule.

- Document code, processes, and best practices to facilitate knowledge sharing and maintainability.

Mandatory Skills :

AWS Services : Proficiency in AWS services such as Lambda, EMR, and Glue for building scalable data solutions.

SQL Expertise : Advanced knowledge of SQL for data manipulation, querying, and analysis.

Python Proficiency : Deep understanding of Python programming for data processing and application development.

Big Data Processing : Experience with tools like Pyspark for processing and analyzing large-scale data sets.

Terabytes : Experience working with very large-scale data sets measured in Terabytes.

Additional Keywords : Familiarity with concepts like map reduce, list comprehensions, and Python lambda functions.

Good to have :

Tools : Experience with project management tools such as JIRA, version control systems like Bitbucket, and documentation tools like Confluence.

AWS Security : Understanding of security best practices in AWS, including API Gateway and IAM.

Design Patterns : Knowledge of software design patterns for building scalable and maintainable data solutions.

Alternatives : Familiarity with alternative tools such as Azure Data Factory, Talend, and Hadoop.

(ref:hirist.tech)
  • Data Engineer

    1 week ago


    Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer.Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry. Our endeavor is to leverage experience...

  • Data Engineer

    1 month ago


    Bangalore, Karnataka, India D Square Consulting Service Full time

    Role : Sr. Data Engineer DOMOExperience : 5+ YearsLocation : Bangalore- Candidates who can join within 30 days, may only applyWorking Mode : Work From OfficeJob Highlights :Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital marketing projects.- Strong knowledge of digital marketing toolsPreferred...

  • Python Developer

    1 month ago


    Bangalore, Karnataka, India IT Full time

    Job Overview :We are looking for a seasoned Python Developer with over 10 years of experience in IT, with a focus on data engineering. The ideal candidate will have extensive knowledge of Python, Pyspark, Azure Databricks, ADF, SQL, and data warehouse concepts. This role requires strong technical skills, project management capabilities, and excellent...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India GLAXOSMITHKLINE ASIA PVT. LTD Full time

    A new world-leading consumer health company. Shaped by all who join us. Together, we're improving everyday health for billions of people. By growing and innovating our global portfolio of category-leading brands including Sensodyne, Panadol, Advil, Voltaren, Theraflu, Otrivin, and Centrum through a unique combination of deep human understanding and...

  • Juego Studios

    3 weeks ago


    Bangalore, Karnataka, India Juego Studios Full time

    As a Python Data Engineer for Test at Client you will be at the forefront of ensuring the quality and reliability of our data systems. You will play a critical role in analyzing raw data, building test frameworks, and validating data products using Python. Collaborating closely with data analytics experts and stakeholders, you will contribute to the...


  • Bangalore, Karnataka, India Pluralsight Full time

    Job Description : Working at Pluralsight Founded in 2004 and trusted by Fortune 500 companies, Pluralsight is the technology skills platform organizations and individuals in 150+ countries count on to create progress for the world. Our platform helps technologists master their craft and take control of their careers. We empower businesses everywhere to build...

  • Data Engineer

    3 weeks ago


    Bangalore/Hyderabad, Karnataka, India Indium Software Full time

    Job Description:We are seeking a talented and motivated Data Engineer to join our team. The ideal candidate should have a strong background in SQL, Python, ETL, and data visualization, with experience working closely with stakeholders and providing data support to Data Analysts. As a Data Engineer, you will play a key role in developing data pipelines,...


  • Bangalore, Karnataka, India BOOTLABS TECHNOLOGIES PRIVATE LIMITED Full time

    Responsibility :- At least 4 years of experience in AWS technologiesThe documented ability of delivering innovative solutions in an applied environment related to at least one of the following areas :- ETL, Data pipeline, Workflow- Strong knowledge of programming in Python, Java, or Scala.- High level of competence in global and cross-skilled collaborative...

  • Engineering Manager

    1 month ago


    Bangalore, Karnataka, India Freelancer Recruiter Full time

    Mandatory requirements :- Apache Kafka- Elastic search- Spark Spring- NoSql- Swift- Distributed systems and Java (preferred)- Only from product organisationsResponsibilities : - Manage software development projects by setting requirements, goals and timelines. - Create clear career paths for team members and help them grow with regular & deep mentoring....


  • Bangalore, Karnataka, India BOOTLABS TECHNOLOGIES PRIVATE LIMITED Full time

    Job Description :- At least 3 years of experience in AWS technologies- The documented ability of delivering innovative solutions in an applied environment related to at least one of the following areas : ETL, Data pipeline, Workflow- Strong knowledge of programming in Python, Java, or Scala.- High level of competence in global and cross-skilled collaborative...

  • Principal Engineer

    1 month ago


    Bangalore, Karnataka, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    What Success Will Look Like :- Design, build and maintain the stream processing, time-series analysis system which is at the heart of Data Platform's products- Responsible for architecture of the platform- Build features, enhancements, new services and bug fixing in Scala and Rust on a Jenkins based pipeline to be deployed as Docker containers on...

  • Big Data Engineer

    1 month ago


    Hyderabad/Bangalore, Karnataka, India HRwing Full time

    Role & Responsibilities:Good development practices:Hands on coder with good experience in programming languages like Java or Python.Hands-on experience on the Big Data stack like PySpark, Hbase, Hadoop, Mapreduce and ElasticSearch.Good understanding of programming principles and development practices like checkin policy, unit testing, code deploymentSelf...


  • Bangalore, Karnataka, India Freelancer Recruiter Full time

    Desired Skills & Qualifications :- Deep understanding of algorithms and data structures.- Should have worked with MVC frameworks, web / client based server architecture, structured and unstructured databases.- Ability to translate product spec to engineering actionables.- Ability to navigate and structure code bases.- Rudimentary skills around service...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Varite India Full time

    Duties :- Experience working in Azure/Databricks.- Expert in Python to create application and ETL to extract data from SharePoint, Salesforce, SQL, etc.- Experience working with APIs and integrating data from various sources into Python-based applications.- Experience in SQL along with writing complex queries.- Exposure to C# and Azure function along with...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Recruise Full time

    Job Description :- AWS technologies such as S3, Redshift, Lambda, Athena,Glue, etc, Python, SQL- Bioinformatics or Life science experience Mandate- Programming experience with SQL and Python.- Strong Hands-on experience in built/deploy of end-to-end project.- Experience in ETL processing in AWS cloud, experience in works on orchestration tools preferably...


  • Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    Job Description : We are seeking an experienced and innovative Data Science Manager to lead our team of talented data scientists and machine learning engineers. The ideal candidate will have a strong background in deep learning (DL), machine learning (ML), data science, and computer vision, with a proven track record of delivering data science products in...

  • Data Squad Engineer

    1 month ago


    Bangalore, Karnataka, India Akshaya IT Business Solutions Full time

    Responsibilities : Develop and maintain data pipelines using PySpark for efficient data processing.Utilize AWS services, especially S3 for storage and data processing tasks.Manage data storage and retrieval using HDFS and Hive.Implement Spark for large-scale data processing and analytics.Work with both relational (MySQL) and non-relational (NoSQL) databases...

  • Data Scientist

    1 month ago


    Bangalore, Karnataka, India SGS consulting Full time

    Role : Data Scientist Skills : AI/ML- PythonData visualization tools : Tableau, Matplotlib, SeabornExperience Required : 4-5 yearsLocation : Bangalore (WFO)N.P : Immediate to 15 daysObjectives of this role :- Collaborate with product design and engineering teams to develop an understanding of needs- Research and devise innovative statistical models for data...


  • Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :At Thoucentric, we work on various problem statements.The most popular ones are :- Building capabilities that address a market need, basis our ongoing research efforts- Solving a specific use case for a current or potential client based on challenges on-ground- Developing new systems that help be a better employer and a better partner to...


  • Bangalore, Karnataka, India Infiniti Research Full time

    About Quantzig : . Quantzig is a global analytics and advisory firm with offices in the US, UK, Canada, China, and India. we have assisted our clients across the globe with end-to-end advanced analytics, visual storyboarding, Machine Learning, and data engineering solutions implementation for prudent decision making. We are a rapidly growing organization...