Senior Data Engineer

2 months ago

Pune, India Thinkproject Full time

Training Expert

Want to work in a culture built on mutual trust and respect? How about having the freedom to make work fit into your life (and not the other way round)? A career with Thinkproject could be just the opportunity you're looking for.

What do we do?
Thinkproject is a European market-leader in digitalisation tools for construction companies. It sounds complex, but we'll explain further Construction companies used to use manual administration and physical paperwork for projects (sometimes hundreds of thousands of bits of paperwork for one project). Using our construction intelligence solutions, businesses can go digital, which benefits everyone from the construction companies to the wider public. 
Our mission is to deliver digitalisation to make a safer, healthier and more sustainable AECO (Architecture, Engineering, Construction, Operations) industry. This is a really exciting time to join our company, since our founding in 2000 we have gone from strength-to-strength and have lots of exciting developments coming up soon that you could be a part of.Job Title: Senior Data Engineer (Matillion & Snowflake)Location : PuneContract : PermanentDepartment: R&D

What your day will look like

ETL Pipeline Development: Design, build, and maintain ETL pipelines using Matillion to extract data from SQL databases, transform it according to business rules, and load it into Snowflake. Data Transformation in Snowflake: Write and manage transformations directly within Snowflake using Snowflake Tasks, Streams, and Stored Procedures to handle specific business logic or real-time data requirements. Porting ETL to Snowflake & Spark: Migrate existing ETL pipelines from Matillion into Snowflake and Spark, utilizing the native features of both platforms to optimize data processing, storage, and analytics. Collaborate with Stakeholders: Work closely with data architects, business analysts, and other teams to understand data requirements and ensure that data pipelines are aligned with business needs. Performance Optimization: Continuously optimize ETL and data transformation processes in both Matillion and Snowflake for enhanced performance and reduced operational costs. Monitoring & Troubleshooting: Monitor the health of data pipelines and Snowflake transformations, troubleshooting issues, and implementing solutions to ensure data accuracy and availability. Data Preparation: Implement data cleansing, aggregation, and enrichment processes within Snowflake to prepare data for downstream reporting and analytics. Snowflake Management: Manage and optimize Snowflake's data architecture, ensuring best practices in data modeling, partitioning, and indexing to support analytics and reporting. Compliance & Governance: Ensure all ETL processes and transformations comply with data governance, privacy, and security requirements.

What you need to fulfill the role

Proficiency in Matillion: Extensive experience in building ETL pipelines and transformations using Matillion, extracting data from SQL databases, and loading it into Snowflake. Expertise in Snowflake: Strong understanding of Snowflake architecture, including experience writing and managing Snowflake Data Warehouse, Snowflake Tasks, Streams, and Stored Procedures for real-time data processing and transformations. Additionally, experience optimizing Snowflake billing and credit consumption. SQL Databases: Proven experience working with SQL databases (e.g., MySQL, PostgreSQL, Microsoft SQL Server) and writing complex SQL queries. ETL Design & Optimization: Ability to design scalable ETL processes and optimize the existing ETL processes. Data Transformation: Experience in data cleansing, aggregation, and transformation both in Matillion and directly within Snowflake. Performance Tuning: In-depth knowledge of performance optimization techniques in both Matillion and Snowflake environments, ensuring efficient data processing. Cloud Platforms: Familiarity with Power BI and Azure is an added advantage. Programming Languages: Experience with languages such as Python or Java is a plus. Experience: 5-10 years

What we offer

Lunch 'n' Learn Sessions I Women's Network I LGBTQIA+ Network I Coffee Chat Roulette I Free English Lessons I Thinkproject Academy I Social Events I Volunteering Activities I Open Forum with Leadership Team (Tp Café) I Hybrid working I Unlimited learning

We are a passionate bunch here. To join Thinkproject is to shape what our company becomes. We take feedback from our staff very seriously and give them the tools they need to help us create our fantastic culture of mutual respect. We believe that investing in our staff is crucial to the success of our business.

  • Senior Data Engineer

    6 months ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Protection Engineer

    2 weeks ago

    Pune, Maharashtra, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Data Protection Engineer to join our team at NTT DATA. This is an exciting opportunity to work with a leading global innovator of business and technology services.Key Responsibilities:Implement and maintain data protection systems, ensuring high levels of security and efficiency.Configure and manage data...

  • Data Analytics Engineer

    6 days ago

    Pune, Maharashtra, India Data Dynamics Full time

    About Data Dynamics">Data Dynamics is a leading data-driven company that leverages cutting-edge technologies to drive business growth. Our team of experts collaborates with clients across various industries to deliver innovative solutions.Job Summary">We are seeking an experienced Senior Machine Learning Specialist to join our team. As a key member, you will...

  • Senior Data Scientist

    6 months ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the USA. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    4 weeks ago

    pune, India Antal International Full time

    Senior Software Engineer - Data Engineer As a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Senior data engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data Engineer As a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling...

  • Senior data engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data EngineerAs a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data EngineerAs a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Senior Quality Engineer

    4 weeks ago

    Pune, India NTT DATA Full time

    Job Description Req ID: 302026 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Quality Engineer to join our team in Pune, Mahārāshtra (IN-MH), India (IN).Senior Quality...

  • Data Engineer

    1 month ago

    Pune, India NTT DATA Full time

    Job Description Req ID: 297728 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Pune, Mahārāshtra (IN-MH), India (IN).Job Title Developer/ Data...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data EngineerAs a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data Engineer As a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Senior Data Engineer

    4 weeks ago

    Pune, India Antal International Full time

    Senior Software Engineer - Data EngineerAs a Senior Software Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining data pipelines, data processing applications, and data integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep...

  • Data Scientist

    6 months ago

    Pune, Maharashtra, India Data Dynamics Full time

    **Objectives of this role**: - Collaborate with product design and engineering teams to develop an understanding of needs. - Research and devise innovative statistical models for data analysis. - Build innovative models and integrate them in our product. - Communicate findings to all stakeholders. - Keep current with technical and industry...

  • Senior Network Engineer and Technical Innovator

    4 weeks ago

    Pune, Maharashtra, India NTT DATA Full time

    About the RoleNTT DATA is seeking a highly skilled Senior Network Engineer to join our team. As a key member of our network management services, you will be responsible for ensuring the smooth operation of our clients' IT infrastructure.Your Day at NTT DATAThe role of a Networking Managed Services Engineer (L2) involves proactively monitoring, identifying,...

  • Senior Data Engineer

    4 weeks ago

    pune, India RS Consulting Full time

    Job Title: Senior Data Engineer | Data Engineer Location: Pune Experience Level: 3 - 9 Years Company Type: AI/ML startup Salary Range: Up to 30 LPA (Max) commensurate with experience and skills. No. of Position Open: 10 Please apply only if you are comfortable working with Early stage startup. Qualifications: 3+ years of experience in Data Engineering,...

  • Principal Data Architect

    3 weeks ago

    Pune, Maharashtra, India Data Axle Full time

    About Data Axle:Data Axle Inc. is a 50-year-old industry leader in data, marketing solutions, sales, and research in the US.We have set up a strategic global centre of excellence in Pune, delivering mission-critical data services to our global customers powered by our proprietary cloud-based technology platform and leveraging our proprietary business &...

  • Senior Data Engineer

    3 weeks ago

    Pune, Maharashtra, India PubMatic Full time

    Join PubMatic as a Senior Data Engineer and be part of our team developing cutting-edge data analytics solutions. We are looking for an experienced professional to join our engineering team.About the RoleWe are seeking a skilled Senior Data Engineer to lead the development of our data analytics platform. The ideal candidate will have experience in big data...

  • Senior data engineer

    4 weeks ago

    Pune, India RS Consulting Full time

    Job Title: Senior Data Engineer | Data EngineerLocation: PuneExperience Level: 3 - 9 YearsCompany Type: AI/ML startupSalary Range: Up to 30 LPA (Max) commensurate with experience and skills.No. of Position Open: 10Please apply only if you are comfortable working with Early stage startup.Qualifications:3+ years of experience in Data Engineering, with a...

  • Senior Data Engineer

    4 weeks ago

    pune, India RS Consulting Full time

    Job Title: Senior Data Engineer | Data EngineerLocation:PuneExperience Level:3 - 9 YearsCompany Type:AI/ML startupSalary Range: Up to 30 LPA (Max) commensurate with experience and skills.No. of Position Open: 10Please apply only if you are comfortable working with Early stage startup.Qualifications:3+ years of experience in Data Engineering, with a strong...