CGI - Data Engineer - Data Pipeline

2 weeks ago


BangaloreHyderabadChennaiPune, IN CGI Information Systems and Management Consultants Full time

Required Skills :

- 5+ years of experience in database administration duties

- Bachelor's degree (or equivalent) in computer science, information technology, engineering, or related discipline

- Experience in building or maintaining ETL processes.

- Experienced in working with big data analytics tools.

- Need to be proficient in various database types and technologies, such as PostgreSQL, MongoDB, and Cassandra.

- Familiarity with data orchestration tools like Airflow

- Experience with Azure Data Analytics Architecture preferred.

- Experience working with data sourced from REST and/or SOAP APIs preferred.

- Proven track of requirements gathering and analysis, design, development, coding, testing, and implementation providing support for real-time low latency, distributed software systems processing big data sets

- A successful history of manipulating, processing and extracting value from large, disconnected datasets.

- Communication skills, especially for explaining technical concepts to nontechnical business leaders.

What You'll Do :

- Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.

- Help streamline our data science workflows, adding value to our product offerings and building out the customer lifecycle and retention models.

- Writing and optimizing code for data transformation. This could involve cleaning the data, handling missing values, and transforming raw data into a more usable format for analysis or machine learning tasks.

- Ensuring that data pipelines are running smoothly is crucial. You'll monitor performance metrics, troubleshoot any issues, and work on pipeline optimization to improve efficiency and reduce latency.

- Performing statistical analysis and using results to improve models.

- Participate in designing and implementing RESTful services and microservices

- Define guidelines and best practices to drive consistency across client implementation and make sure consistency is followed across the board.

- Installs, supports, upgrades, and maintains database environments like MongoDB and Postgres

- Supports database best practices relative to tuning, object creation, monitoring, security, logging, and auditing techniques while sharing knowledge and collaborating with others in IT to deliver effective application solutions in a solid operational environment

- Assist in developing best BI and data services deployment practices such as Azure Data Factories and Azure CI/CD pipelines, including development, governance and maintenance

- Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.

- Ensure the quality, security, reliability, and compliance of our solutions by applying our digital principles and implementing both functional and non-functional requirements.

(ref:hirist.tech)
  • Senior Data Engineer

    2 weeks ago


    Bangalore/Hyderabad/Pune, IN LAGNUS PRIVATE LIMITED Full time

    About the job : ROLE : Snowflake and Matillion Data Engineer JD : Experience : 4 to 6 Years Overview :- Design, implement, and optimize data pipelines and workflows using Snowflake and Matillion to support our data integration, transformation, and analytics initiatives.- Collaborate with cross-functional teams to gather requirements, define data models, and...

  • Lead Data Engineer

    2 days ago


    Bangalore/Pune/Hyderabad/Chennai, IN Nilasu consulting Full time

    We are seeking a highly skilled and experienced Data Engineer with a passion for the healthcare industry to join our growing team!In this pivotal role, you will play a key role in building and maintaining our data lake infrastructure. You will leverage your expertise in data pipelines, cloud technologies, and healthcare data to design, develop, and deploy...

  • Cloud Data Engineer

    2 weeks ago


    Bangalore/Hyderabad/Chennai/Pune/Anywhere in India/Multiple Locations, IN Amk technologies Full time

    Job Description :We are seeking a talented and motivated Cloud Data Engineer to join our dynamic team.As a Cloud Data Engineer, you will be responsible for designing, implementing, and maintaining data pipelines and infrastructure on Azure Cloud Platform. You will work closely with cross-functional teams to ensure efficient data flow and integration across...


  • Pune/Ahmedabad/Bangalore/Indore, IN eInfochips Full time

    Role : AWS Data EngineerExperience : 5+ yearsLocation : Pune/Ahmedabad/Indore/ bangaloreCompany Overview :eInfochips, an Arrow company, is a global leader in product engineering and semiconductor design services. Renowned for our technological innovations, we've developed over 500 products, achieving 40M deployments across 140 countries. Our expertise...

  • Python Data Engineer

    2 weeks ago


    Bangalore/Hyderabad/Pune/Mumbai/Chennai, IN Swift Strategic Staff Solutions INC Full time

    Job Description :We are seeking a talented and experienced Python Data Engineer to join our data team and play a crucial role in building, maintaining, and optimizing data pipelines for our cloud-based data warehouse.Responsibilities :- Design, develop, and implement data pipelines for ingesting, transforming, and loading data into Snowflake using Python...


  • Pune/Bangalore/Chennai/Hyderabad, IN CGI Information Systems and Management Consultants Full time

    Job Description :- Experience in application development experience in Scala, Spark, scala test/Junit libraries with maven.- Experience on Hive, Spark using Cloudera data platform.- Knowledge of performance optimization on Hadoop platform, Spark and Scala.- Ability to write optimized SQL queries.- Ability to learn & adapt.- Ability to communicate in clear &...


  • Hyderabad/Bangalore/Chennai, IN CGI Full time

    Job Description :- 5+ years of hands on experience with /designing/development of Informatica tools.- Strong database knowledge Oracle PL/SQL, tuning- Good knowledge on UNIX scripting- Able to write complex relational Oracle/SQL server queries to help with data validation- Expert in writing Oracle procedures, functions, packages and debugging the code.- Good...


  • Hyderabad/Bangalore/Chennai, IN CGI Full time

    Job Description :- Extensive experience in developing, maintaining, and managing Tableau-driven dashboards and analytics.- Strong experience with Tableau for creating dashboards and reports, understanding of Tableau's JavaScript API.- Experience in designing RESTful API services and integrating with third-party services and APIs- Familiarity with...


  • Pune/Bangalore, IN Tekfortune IT India Pvt Ltd Full time

    Title : Databricks EngineerExperience : 4 to 10 yearsWork Location : Pune and Bangalore LocationNotice Period : Immediate to 45 DaysJob Description : We are seeking a highly skilled and experienced Databricks Engineer to join our growing team and play a crucial role in enhancing our data processing and analytics capabilities.As a Databricks Engineer you will...


  • Hyderabad/Bangalore/Chennai, IN CGI Full time

    Job Description : - SME in ServiceNow system applications and custom applications.- Minimum 5 years of ServiceNow platform experience- Create, maintain, and enhance Incident Management, Problem Management, Service Request Management, Service Catalog, Knowledge Management, Self-Service Portals- Create and configure Business Rules, UI Policies, UI Actions,...

  • Data Engineer

    2 weeks ago


    Pune/Mumbai/Hyderabad/Chennai/Bangalore, IN RapidBraiins Full time

    Position Overview :As a Talend - Data Engineering Specialist, you will play a key role in designing, developing, and maintaining data pipelines using Talend, AWS, and SQL. You will work closely with cross-functional teams to gather requirements, design data solutions, and implement scalable and efficient ETL processes. Additionally, you will leverage your...


  • Bangalore/Chennai/Pune/Hyderabad, IN CGI Information Systems and Management Consultants Full time

    Job Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Integrating new data management tools into company ecosystems- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for...

  • Data Engineer

    2 weeks ago


    Bangalore/Chennai/Hyderabad, IN Cortex Consulting Pvt. Ltd. Full time

    Responsibilities :- Design, construct, install, test, and maintain highly scalable data management systems.- Develop and implement ETL processes to collect, store, and process data from various sources.- Build and optimize data pipelines for efficient data ingestion and transformation.- Collaborate with data analysts and data scientists to understand data...

  • GCP Developer

    3 days ago


    Bangalore/Pune/Noida/Hyderabad/Chennai, IN Nilasu consulting Full time

    JOB DESCRIPTION:Must Have: - Min 3 years' Experience with Building Pipelines using GCP-Big-Query - Min 2-3 Experience in working on Large Scale Data warehouses like Teradata - Experience building and optimizing data warehousing data pipelines (ELT and ETL), architectures and data sets. - Advanced working SQL knowledge and experience working with...

  • Data Engineer

    2 weeks ago


    Bangalore/Any Location, IN INSIGHT GLOBAL SOLUTIONS Full time

    Data Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...


  • Delhi NCR/Metros/Any Location/Hyderabad/Guwahati/Gurgaon/Gurugram, IN TOTL RPO SOLUTIONS LLP Full time

    job Description :As a member of the data engineering team, you will be the key technical expert developing and overseeing data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business.You'll be an empowered member of a team of data engineers who build data pipelines into...

  • Data Engineer

    2 weeks ago


    Bangalore/Noida, IN Xperthunt Full time

    About the Role :We're seeking a highly skilled Data Engineer to join our team and play a pivotal role in building and maintaining our data infrastructure.You'll collaborate closely with data scientists, analysts, and stakeholders to understand their data needs and design, develop, and implement data solutions that empower effective analysis and...


  • Bangalore/Anywhere in India/Multiple Locations, IN Info Origin Inc Full time

    We are seeking a highly skilled and experienced Data Warehouse Architect to join our team! In this pivotal role, you will play a key role in building and maintaining our data warehouse infrastructure, with a specific focus on integrating data from Workday. You will leverage your expertise in data warehousing, ETL development, and Workday technologies to...

  • Data Engineer

    2 weeks ago


    Bangalore/Pune, IN Pylon Management Consulting Full time

    Requirements:- Overall 5+ years of IT experience with 3+ years in data-related technologies.- Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP).- Hands-on experience with the Hadoop stack - HDFS, sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink,...

  • Azure Data Engineer

    6 days ago


    Bangalore/Hyderabad/Pune, IN Kezan Consulting Full time

    Work Timings : 10 AM - 7 PMNumber of Openings : 5Joining Date : Within a weekJob Description :We are seeking experienced Azure Data Engineers to join our team on a contractual basis. As an Azure Data Engineer, you will be responsible for designing, implementing, and maintaining data solutions on the Azure platform. The ideal candidate will have a strong...