Senior Data Engineer

3 weeks ago

Pune, India Edgematics Full time
About Edgematics:Edgematics Technologies is a leading global consulting and solutions integrator in the field of data management, business intelligence, and analytics. We specialize in delivering end-to-end data management solutions to help organizations make data-driven decisions and achieve their business objectives.

To know more about Edgematics visit our website - this role:We are seeking a talented and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will be passionate about data and possess strong technical skills to design, implement, and maintain scalable data pipelines. This role offers an exciting opportunity to work on cutting-edge technologies and drive innovation in data engineering within our organization. We are looking for a talented and experienced Data Engineer with hands-on expertise in any ETL Tool with full knowledge about CI/CD practices with leading a team technically more than 5 and client facing and create Data Engineering, Data Quality frameworks. As a Data Engineer you must ensure to build ETL jobs, Data Quality Jobs, Big Data Jobs performed performance optimization by understanding the requirements, create re-usable assets and able to perform production deployment.

ResponsibilitiesWork with a team of engineers in designing, developing, and maintaining scalable and efficient data solutions using Any Data Integration (like Informatica, Talend, Pentaho, DataStage etc.) and any Big Data technologies.Design, develop, and maintain end-to-end data pipelines using Any ETL Data Integration (like Informatica, Talend, Pentaho, DataStage etc.) to ingest, process, and transform large volumes of data from heterogeneous sources.Implement database solutions for storing, processing, and querying large volumes of structured and unstructured data.Integrate and optimize data flows between various databases, data warehouses, and Big Data platforms.Collaborate with cross-functional teams to gather data requirements and translate them into scalable and efficient data solutions.Optimize ETL, Data Load performance, scalability, and cost-effectiveness through optimization techniques.Interact with Client on daily basis and provide technical progress and respond to technical questions.Implement best practices for data integration.Implement complex ETL, Big Data solutions using technologies such as Hadoop, Apache Spark, or similar frameworks to process and analyze massive datasets.Ensure data quality, reliability, and security across all stages of the data pipeline.Troubleshoot and debug data-related issues in production systems and provide timely resolution.Stay current with emerging technologies and industry trends in data engineering technologies, CI/CD and Big Data, and incorporate them into our data architecture and processes.Optimize data processing workflows and infrastructure for performance, scalability, and cost-effectiveness.Provide technical guidance and foster a culture of continuous learning and improvement.Implement and automate CI/CD pipelines for data engineering workflows, including testing, deployment, and monitoring.Perform migration to production deployment from lower environments, test & validate.

Must Have SkillsBachelor’s or master’s degree in computer science, Engineering, or a related field.Certified in Any ETL Technologies, Cloud Data Engineering, Databases like Snowflake.5 years of experience in software engineering or a related role, with a strong focus on Any ETL Tool, database, integration, and Big Data technologies.Proficiency in Any ETL tools like Talend, Informatica etc. for Data Integration for building and orchestrating data pipelines.Hands-on experience with relational databases such as MySQL, PostgreSQL, or Oracle, and NoSQL databases such as MongoDB, Cassandra, or Redis.Solid understanding of database design principles, data modeling, and SQL query optimization.Experience working with Big Data technologies such as Hadoop, Apache Spark, and Apache Hive.Experience with data warehousing, Data Lake , Delta Lake concepts and technologies, data modeling, and relational databases.Familiarity with tools and platforms for CI/CD, such as Jenkins, GitLab CI/CD, or CircleCI.Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.Strong understanding of software development lifecycle (SDLC) and CI/CD best practices.Excellent problem-solving skills and the ability to work independently as well as part of a team.Strong programming skills in languages such as Java, Python, or Scala.Excellent leadership, communication, and interpersonal skills, with the ability to lead and motivate a team.

Good to have skills:Data warehousingo

Strong Data warehousing conceptso

Strong Data Modelling skills (analytical models), preferably with ERWino

Must have implemented end-to-end DWH projects (discovery, requirements gathering, HLD/LLD documentation, development, testing, SIT, UAT, deployment, pre and post prod checks etc.)o

Must have worked with Teradata / Greenplum / Synapse / Big Query / Snowflake

Data Integration / Data Engineeringo

Strong logical and analytics abilities in DI/DQo

Very strong skills in writing logical code for data engineering, which has re-usable capabilities without rework.o

Strong knowledge and technical expertise in ETL tools such as Talend / Informatica / DataStage / Pentaho etc.

Data Qualityo

Strong understanding and experience in DQ frameworkso

Must have implemented atleast 2 projects involving end-to-end DQ frameworks.o

Very strong skills in writing logical code for data quality, which has re-usable capabilities without rework.

  • Senior Data Engineer

    2 days ago

    Pune, India Data Axle Full time

    Job DescriptionAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...

  • Senior Data Engineer

    1 week ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Scientist

    4 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the USA. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Scientist

    1 week ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the USA. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based...

  • Senior Data Engineer

    2 weeks ago

    Pune, India timesjobs Full time

    Senior Data EngineerPosition: Senior Data EngineerExperience: 8 yearsJob Location: PuneSkills Required:Developing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environmentsDevelopment experience in one or more object-oriented programming languages (e.g., Py-spark, Python, Java, C )Experience of working...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India timesjobs Full time

    Senior Data EngineerPosition: Senior Data EngineerExperience: 8 yearsJob Location: PuneSkills Required:Developing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environmentsDevelopment experience in one or more object-oriented programming languages (e.g., Py-spark, Python, Java, C )Experience of working...

  • Senior Data Engineer

    3 weeks ago

    Pune, India Siemens Energy Full time

    Senior Data EngineerAbout the RoleLocationIndia Maharashtra Pune Company Siemens Energy Industrial Turbomachinery India Private Limited Organization Transformation of Industry Business Unit Compression Full / Part time Full-time Experience Level Mid-level Professional Profile Overview: Title: Senior Data Engineer Location: Pune, India Role Description:...

  • Senior Data Engineer

    3 weeks ago

    Pune, India Lear Full time

    Job Description: Senior Data Engineer (Palantir Foundry)As aSenior Data Engineerat Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical decision-making...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Siemens Energy Full time

    Senior Data Engineer About the Role Location India Maharashtra Pune Company Siemens Energy Industrial Turbomachinery India Private Limited Organization Transformation of Industry Business Unit Compression Full / Part time Full-time Experience Level Mid-level Professional Profile Overview: Title: Senior Data Engineer Location: Pune, India Role...

  • Senior Data Engineer

    4 weeks ago

    pune, India Siemens Energy Full time

    Senior Data Engineer About the Role Location India Maharashtra Pune Company Siemens Energy Industrial Turbomachinery India Private Limited Organization Transformation of Industry Business Unit Compression Full / Part time Full-time Experience Level Mid-level Professional Profile Overview: Title: Senior Data Engineer Location: Pune,...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Procore Technologies Full time

    Job DescriptionWe are looking for a highly skilled and experiencedSenior Data Engineerto join our team and help us build and maintain an advanced data platform designed to facilitate data-driven decision-making in the construction industry. TheSenior Data Engineerintegrates data from multiple sources, including third parties, to provide a unified and...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Lear Full time

    Job Description: Senior Data Engineer (Palantir Foundry) As a Senior Data Engineer at Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical decision-making...

  • Senior Data Engineer

    4 weeks ago

    pune, India Lear Full time

    Job Description: Senior Data Engineer (Palantir Foundry) As a Senior Data Engineer at Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical...

  • Senior Data Engineer

    2 weeks ago

    pune, India Procore Technologies Full time

    Job DescriptionWe are looking for a highly skilled and experienced Senior Data Engineer to join our team and help us build and maintain an advanced data platform designed to facilitate data-driven decision-making in the construction industry. The Senior Data Engineer integrates data from multiple sources, including third parties, to provide a unified and...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Procore Full time

    Job DescriptionWe are looking for a highly skilled and experienced Senior Data Engineer to join our team and help us build and maintain an advanced data platform designed to facilitate data-driven decision-making in the construction industry. The Senior Data Engineer integrates data from multiple sources, including third parties, to provide a unified and...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Procore Technologies Full time

    Job DescriptionWe are looking for a highly skilled and experienced Senior Data Engineer to join our team and help us build and maintain an advanced data platform designed to facilitate data-driven decision-making in the construction industry. The Senior Data Engineer integrates data from multiple sources, including third parties, to provide a unified and...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Saaki, Argus and Averil Consulting Full time

    We are trusted and dependable partners for quality recruitment services for a number of global organizations. We have built long-lasting relationships with clients by providing value-added services every step of the way. At Saaki, Argus and Averil Consulting we understand the key to a successful engagement is to be adaptive to our clients’ business needs....

  • Senior Data Engineer

    4 weeks ago

    pune, India Saaki, Argus and Averil Consulting Full time

    We are trusted and dependable partners for quality recruitment services for a number of global organizations. We have built long-lasting relationships with clients by providing value-added services every step of the way. At Saaki, Argus and Averil Consulting we understand the key to a successful engagement is to be adaptive to our clients’ business needs....

  • Senior Data Engineer

    4 weeks ago

    Pune, India Synechron Full time

    We are hiringSenior Data Engineers (PySpark)at Synechron Technologies.Job Title : Senior Data Engineers (PySpark)Experience : 3+ YearsNotice Period: Looking for immediate to 30 days' joinersJob Location : Hinjewadi, Pune (Hybrid Work Model)Software Requirements:Proficient in Python, PySpark, and SQLExperience in the build and deployment of Big Data...