Data Engineer

4 weeks ago


PuneBangaloreChennai, IN ADVANSOFT Full time

Skills :

- Hadoop

- Python

- Spark

- PySpark

- ETL (Extract, Transform, Load)

Roles & Responsibilities :

- Data Ingestion: Develop and maintain data pipelines for ingesting raw data from various sources into the Hadoop ecosystem.

- Data Processing: Utilize Python and Spark to process and transform large volumes of data efficiently, ensuring scalability and performance.

- Data Modeling: Design and implement data models that facilitate efficient querying and analysis of structured and unstructured data.

- ETL Development: Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and integrate data from different sources into data warehouses or data lakes.

- Performance Tuning: Optimize data processing workflows and queries to improve performance and reduce processing times.

- Data Quality Assurance: Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data.

- Data Governance: Enforce data governance policies and standards to ensure compliance with regulatory requirements and industry best practices.

- Documentation: Maintain documentation for data pipelines, data models, and ETL processes to facilitate knowledge sharing and troubleshooting.

- Collaboration: Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.

- Continuous Improvement: Stay updated with emerging technologies and best practices in data engineering, and proactively identify opportunities for process optimization and automation.

(ref:hirist.tech)
  • Python Data Engineer

    4 weeks ago


    Bangalore/Hyderabad/Pune/Mumbai/Chennai, IN Swift Strategic Staff Solutions INC Full time

    Job Description :We are seeking a talented and experienced Python Data Engineer to join our data team and play a crucial role in building, maintaining, and optimizing data pipelines for our cloud-based data warehouse.Responsibilities :- Design, develop, and implement data pipelines for ingesting, transforming, and loading data into Snowflake using Python...


  • Bangalore/Hyderabad/Chennai/Pune, IN CGI Information Systems and Management Consultants Full time

    Required Skills :- 5+ years of experience in database administration duties- Bachelor's degree (or equivalent) in computer science, information technology, engineering, or related discipline- Experience in building or maintaining ETL processes.- Experienced in working with big data analytics tools.- Need to be proficient in various database types and...

  • AWS Data Engineer

    4 days ago


    Bangalore/Pune, IN Technomech Placements Full time

    This is a full-time hybrid role for a Senior AWS Data Engineer. The Senior AWS Data Engineer will be responsible for day-to-day tasks such as data engineering, data modeling, ETL (Extract Transform Load), data warehousing, and data analytics.Qualifications :- Data Engineering, Data Modeling, and ETL (Extract Transform Load) skills- Data Warehousing and Data...

  • Data Engineer

    4 weeks ago


    Pune/Mumbai/Hyderabad/Chennai/Bangalore, IN RapidBraiins Full time

    Position Overview :As a Talend - Data Engineering Specialist, you will play a key role in designing, developing, and maintaining data pipelines using Talend, AWS, and SQL. You will work closely with cross-functional teams to gather requirements, design data solutions, and implement scalable and efficient ETL processes. Additionally, you will leverage your...

  • Big Data Engineer

    4 weeks ago


    Bangalore/Chennai/Pune, IN ADVANSOFT Full time

    We are hiring for Data engineer with top most MNC client;Exp : 5+yearsNp : immediate to 15daysLocation : & Responsibilities : - 5+ years of working experience in ETL scalable data pipeline usin61Scala, Python, Pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nill, and incremental Data Load with Big data technologies.- Experience working with Databases like...

  • Data Engineer

    4 weeks ago


    Mumbai/Chennai/Pune/Bangalore, IN Coders Brain Pvt Ltd Full time

    JD - Python and Snowflake- Experienced Data engineer(6+) responsible for developing, overseeing, organizing, storing, and analyzing data and data systems.- Responsible for Cloud migrations for data pipelines using AWS and Snowflake- Participate in all aspects of the software development lifecycle for AWS and Snowflake solutions, including planning,...

  • AI/ML Engineer

    4 weeks ago


    Bangalore/Pune/Hyderabad/Chennai, IN Nilasu consulting Full time

    Summary :We are seeking a talented and passionate AI/Machine Learning Engineer to join our team. You will play a key role in the entire lifecycle of machine learning projects, from data engineering and classical machine learning to potentially exploring deep learning and natural language processing techniques. Your strong foundation in Python coding,...

  • Lead Data Engineer

    1 week ago


    Bangalore/Pune/Hyderabad/Chennai, IN Nilasu consulting Full time

    We are seeking a highly skilled and experienced Data Engineer with a passion for the healthcare industry to join our growing team!In this pivotal role, you will play a key role in building and maintaining our data lake infrastructure. You will leverage your expertise in data pipelines, cloud technologies, and healthcare data to design, develop, and deploy...

  • Data Engineer

    4 weeks ago


    Bangalore/Pune, IN vipsa solutions Full time

    SALARY : 15LPA - Engineer:Location: Pune, BangaloreExperience: 6-9 yearsNP: Immediate- 15 daysKey accountabilities:As a Data Engineer you will be accountable for:- Understanding operational needs by collaborating with specialized teams- Supporting key, involves architecture designing, building and deploying data systems, pipelines etc- Build agile, scalable,...

  • AWS Data Engineer

    4 weeks ago


    Bangalore/Pune/Chennai, IN Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • Data Engineer

    4 weeks ago


    Bangalore/Pune, IN Gethyr Full time

    We are seeking a talented and motivated Data Engineer stream processing, particularly utilizing Flink and Kafka, alongside expertise in Spark and AWS technologies. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining robust stream processing solutions to handle large volumes of data in real-time, ensuring high...

  • Data Engineer

    4 weeks ago


    Bangalore/Chennai/Hyderabad, IN Cortex Consulting Pvt. Ltd. Full time

    Responsibilities :- Design, construct, install, test, and maintain highly scalable data management systems.- Develop and implement ETL processes to collect, store, and process data from various sources.- Build and optimize data pipelines for efficient data ingestion and transformation.- Collaborate with data analysts and data scientists to understand data...

  • Data Engineer

    1 day ago


    Hyderabad/Chennai/Pune, IN Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...


  • Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Data Engineer

    4 days ago


    Bangalore/Pune, IN OptOut Full time

    Role Description :- Data modeling and architecture : You would be responsible for designing and implementing data models that are optimized for Azure services, including Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage.- ETL and data integration: You would be responsible for building and maintaining data pipelines that move data from various...

  • Data Engineer

    4 weeks ago


    Mumbai/Bangalore/Chennai/Pune, IN LanceTech Solutions Full time

    Data Engineer - SQL, PLSQL, Snowflake, ETLWe're seeking an experienced Data Engineer with expertise in SQL, strong PLSQL skills, Snowflake, and ETL development. As a key member of our team, you'll design and implement data-driven solutions, collaborating with software developers, database architects, and data scientists. Your role involves building...


  • Bangalore/Pune/Chennai, IN Indiglobe IT Solutions Pvt. Ltd. Full time

    About the jobSkills : Azure, Azure SQL, Azure SynapseWork type: Full-timeMode of Work: Hybrid (3 days WFO)Notice Period: Max 30 daysMust To Have Skills :- Azure with SQL (Azure SQL) combination is a must to have SSIS SSRS with Azure - SQL with Azure is what you need to look at.Qualifications :- Experience in ETL & Data Warehousing- Should have excellent...

  • Senior Data Engineer

    4 weeks ago


    Bangalore/Trivandrum/Thiruvananthapuram/Chennai/Pune, IN Arting Digital Full time

    Senior Data EngineerExperience : 6y+ Location : Trivandrum, Chennai, Pune, Bangalore Budget : 15 - 25 LPA Work Mode : On-SiteSkills : - Ataccama, Syndigo, Snowflake, Databricks- Requirements :- 6+ Years experience as a Data Engineer, who played a key/lead role in implementing one or two large data solutions- Programming experience in Scala or Python, SQL-...

  • Abinitio Developer

    2 weeks ago


    Any Location/Bangalore/Chennai/Pune, IN Align Techne Full time

    Location : Pune/Chennai/ bangaloreMand : GDE, Unix Commands, Air commands, Mdump, MDB, M commands, Any DatabaseTotal Exp : 3 - 10YearsRel Exp : 2+ Years Mandatory Skills for screening : - ETL Development : Lead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy,...

  • Data Engineer

    4 weeks ago


    Bangalore/Pune, IN Pylon Management Consulting Full time

    Requirements:- Overall 5+ years of IT experience with 3+ years in data-related technologies.- Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP).- Hands-on experience with the Hadoop stack - HDFS, sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink,...